betaFeedback

WorksComputer Science

LoStBench: Benchmarking Longitudinal Studies Data Integration

Abstract

A longitudinal study observes the same subjects or variables, repeatedly over an extended period of time, allowing researchers to examine changes, trends, and relationships as they develop. Such studies typically evolve over their life cycles, as improved understanding of the study's domain, or external factors trigger the need for revisions. The resulting heterogeneity among collected data must be resolved before the survey's results may be analyzed. Although this problem is a form of schema matching, it is a form of schema matching that differs significantly from prior workloads explored by the database community. For instance, rather than the short attribute names and data instances that many schema matching tools rely on, a stronger signal in longitudinal studies arises from longer prose questions that these tools are less equipped to handle. This paper documents the joint efforts of a team of computer scientists and domain experts to adapt existing schema matching algorithms to our new use case of longitudinal data integration. We present a case study in the context of a representative workload: The American National Election Studies (ANES) dataset, a nearly eight-decade survey with publicly available data and a canonical domain-expert curated schema matching. Using ANES, we develop LoStBench (Longitudinal Studies Benchmark for data integration), a medium-scale, subject-matter-expert-annotated benchmark. Using LoStBench, we conduct a comprehensive empirical study evaluating state-of-the-art schema matching algorithms, and expose limitations of these algorithms when applied in the context of longitudinal studies. We propose improvements that address some of these limitations, and identify key challenges to motivate future data integration work to address the others.

Introduction

A longitudinal study observes the same subjects or variables, repeatedly over an extended period of time, allowing researchers to examine changes, trends, and relationships as they develop. Survey-based longitudinal studies, where researchers poll participants using a questionnaire, or "survey instrument" arise in a wide range of research disciplines. A common feature of longitudinal studies is that they evolve over time. For example, revisions to the study may be triggered when researchers develop a deeper understanding of the study domain that renders a question invalid or ambiguous; or when decades-long shifts in cultural norms affect the interpretation of a question. As a survey instrument is revised, the data collected across multiple revisions becomes heterogeneous; each revision of the study produces a dataset with a distinct schema. To analyze the resulting data, researchers expend significant manual effort integrating data collected across the survey revisions.

This is, at first glance, a classical data integration problem: We want to map a collection of datasets with distinct (local) schemas into a uniform (global) schema. More concretely, this is a problem of schema matching, or establishing correspondences between attributes in related schemas. Schema matching is addressed as part of data integration [2, 6, 22, 42], table enrichment [51] and semantic type detection [29]. There is potential in existing tools to be a good fit for longitudinal studies as well.

The authors of this paper include experts from diverse set of disciplines, with experience conducting longitudinal studies in the fields of linguistics, environmental health, and medicine. This paper documents our ongoing1 effort to adapt existing schema matching techniques into the context of longitudinal studies. To focus this discussion, and to make it easier for others to build on our efforts, we implemented a case study using a dataset from The American National Election Studies (ANES)[1], a longitudinal study that has been running for nearly eight decades. ANES is ideal, because (i) it captures many of the challenges that we have faced in our own longitudinal data integration efforts, and (ii) its codebook defines a hand-curated ground truth schema matching. Concretely, this paper presents the Longitudinal Study Benchmark (LoStBench), a benchmark based on the ANES schemas, datasets, and codebook.2 We explore and address the challenges of adapting algorithms for schema matching and related problems, both to ANES specifically, and to longitudinal studies in general, and then evaluate a wide range of existing algorithms on this new and rather distinct use case.

Figure 1. A sample of flow of related variables across years in the ANES dataset [1]. Rectangular Nodes are dataset versions annotated with the number of variables in that version. We show the flow between adjacent and every second versions only for clarity in the figure. The curved flow joining two adjacent versions represent the number of variables present in both the versions. ANES alternates pre- and post- election questionnaires, and so the remaining lines link surveys of the same type.

Although the ANES data and schema share many statistical properties with common schema matching benchmarks, several key differences affect the effectiveness of existing matching algorithms. First, while ANES provides attribute names for each of its schemas, the prose of the questions provides a far richer signal to mine for attribute matchings. We found that schema matching algorithms that can effectively incorporate this prose, for example via word embeddings, to be substantially more effective. Similarly, compared to other schema matching problems, ANES features a relatively low level of data variation due to common, repeating question patterns, as well as a rather high null response rate due to unanswered questions, or frequent use of schema denormalization. As a result, we found that instance-based techniques perform more poorly than metadata-based techniques. Finally, some iterations of the ANES ask as many as 2000 questions. Such wide schemas pose scalability problems for schema integration algorithms intended for use with dozens, or at most hundreds of attributes. We propose a blocking-style approach to schema integration based on table unionability. Finally, we propose and evaluate an algorithm to leverage the sequential nature of longitudinal datasets.

Longitudinal Data Management: In Practice

The data collection and management problems of longitudinal studies are relevant to most research that involves collecting data on human social behavior, especially outside of highly controlled settings. They are especially common for surveys and interviews, when a researcher enters data based on interviewee responses, or when the data is entered directly by study participants. Among the authors of this paper are researchers from three distinct domains: linguistics, environmental health, and healthcare. Although we use ANES as a representative use case for our benchmark due to its publicly available ground truth, we motivate the need for data integration techniques specialized for longitudinal data by outlining concrete challenges that we have experienced.

Linguistic Research. Linguistic research on underdescribed languages typically involves a linguist interacting with users of an underdescribed language to gather data about the language and its users [12, 28, 70]. Alongside recordings of both structured interactions and naturalistic examples of language use, sociolinguistic interviews are often used to gather information about the linguistic background of individuals from whom data has been collected. A questionnaire or similar instrument is often used to conduct these interviews, and they are administered by members of the research team. One of our authors has been conducting a multi-year linguistic study in Cameroon, which has required multiple revisions to the data collection protocol. Some changes are necessary to cope with exogenous factors like political conflict: e.g., “What village are you from?” becomes “What village did you live in before being displaced?” Conversely, other changes result from improved domain understanding: e.g., A discovery that some individuals use different names when interacting with different people required the survey to be adjusted to allow for an individual to provide multiple names and the contexts in which they use them.

Environmental Health Studies. Pollution in an individual’s environment is frequently correlated with social factors like the individual’s role in the family (e.g., male vs. female caregiver), income, resources or social position, or the attributes of the neighborhood where the family resides. Another of our authors has been directing a research program in Montevideo, Uruguay since 2009, collecting data on individual, family, and neighborhood-level factors relating to exposure metrics and health effects. The current round of data collection, with annual surveys between 2021 and 2024, involves 15 survey instruments posed to caregivers and youth totaling over 250 questions. An evolving understanding or operationalization of social factor variables (e.g., differential income distribution due to inflation or the changing availability of different types of items or possessions like smartphones that may be used to calculate asset-based household socioeconomic status score), and changes in exposure patterns and/or definitions (e.g., better understanding of the exposure sources or patterns for different environmental chemicals), have necessitated changes in the wording of questions, including question removal, addition, and alteration. Over time, survey respondents evolve as well: Initially, when study participants were children, the caregivers served as primary study informants. Over time, responding has shifted to the youth, who can report on their own behaviors or states (e.g., mental health). These shifts require appropriate tools to annotate, combine, and analyze individual question responses as well as any derived indices.

Treatment for Hepatitis-C. Social, behavioral, and environmental factors contribute to health inequalities and play a disproportionately large role in health outcomes. Such factors contribute substantially to an individual’s overall physical and mental health, and can have a deleterious effect on health outcomes [16]. Specifically, low literacy, racial segregation, poverty, low socioeconomic support, and low levels of social support contribute substantially to mortality [27]. Recently, professional societies, such as the American College of Physicians (ACP), have highlighted research gaps in the area of SDOH based upon the realization that they require prioritization in order to improve health outcomes, particularly among disadvantaged and under-served populations [16, 69, 71]. A third author is working on a project to understand the relationship between substance abuse and social factors, as part of research on improving treatment for Hepatitis-C. A key source of data for this project is patient in-take forms from methadone clinics collected across multiple care providers, and over the course of decades. Each care provider asks a different, though often related, set of questions. Moreover, even within a single care provider, the structure of the forms evolves over time, with questions added, removed, and/or rephrased.

Overview

The balance of this paper is organized as follows. In § 2, we formally define the problem of schema matching and put past work in schema matching and similar problems into a shared context. § 3 presents our approach to adapting schema matching algorithms to longitudinal data integration. § 4 and § 5 outline LoStBench, first by reviewing the ANES dataset, and then our experimental goals for the benchmark. § 6 presents the results of our evaluation, and we conclude in § 7.

Preliminaries

Data integration concerns combining data from multiple heterogeneous sources to provide a unified view that can serve as a foundation for downstream analytics [36]. Despite its seemingly straightforward definition, data integration involves significant challenges, including system-level issues, logical heterogeneity across sources, and administrative barriers such as siloed data ownership [22, 68].

While integration objectives may vary depending on the workload, a fundamental subtask is schema matching, the identification of correspondences and relationships across datasets and their attributes. Schema matching is a core component of data integration and arises in numerous applications, including data warehousing, e-business, and semantic query processing [59]. In such a task, we are generally given 2 “source” dataset instances with schemas S1S_1, S2S_2, where each schema SiS_i is a set of attributes a∈Sia \in S_i. We are told that these datasets correspond to the same logical entity, and so we would like to connect the attributes of the two schemas.

Naively, we can construct a unified “target” dataset with schema STS^T, and then establish partial mapping functions Hi:Si→ST\mathcal{H}_i : S_i \to S^T from the source schemas to the target. However, the target schema is often unknown at first, and so an equivalent framing of the problem is to discover a set of pairwise attribute matchings M\mathcal{M} such that every bi-clique in M\mathcal{M} corresponds to a target attribute.

M={⟨a,a′⟩∣a∈S1,a′∈S2,H1(a)=H2(a′)}\mathcal{M} = \{ \langle a, a' \rangle\mid a \in S_1, a' \in S_2, \mathcal{H}_1(a) = \mathcal{H}_2(a') \}

Schema matching algorithms infer M\mathcal{M} by leveraging properties of each attribute, which may include schema-level metadata [20, 41, 48, 56, 73], instance-level data [73, 75], or a combination of the two [4, 33, 38]. Regardless of the specific properties leveraged, the typical approach is to define an easily computable distance measure3^{3} over pairs of attributes DIST⁡(a,a′)\operatorname{DIST}(a, a'). Ideally, for the ground truth matching MG\mathcal{M}^G, lower values of the measure should correlate with a greater chance that ⟨a,a′⟩∈MG\langle a, a' \rangle\in\mathcal{M}^G for a given pair a∈Sia \in S_i, a′∈Sja' \in S_j. When aligning schemas, we are additionally interested in two ranking schemes derived from the distance measure.

Definition 2.1 (Attribute Rank). Given an attribute a∈S1a \in S_1, the attribute rank, denoted ARANK⁡a(a′)\operatorname{ARANK}_a(a') is defined with respect to the list of attributes a′∈S2a' \in S_2 that could potentially match with aa:

ARANK⁡a(a′)=∣{a′′∣DIST⁡(a,a′′)≤DIST⁡(a,a′)}∣\operatorname{ARANK}_a(a') = |\{a'' \mid\operatorname{DIST}(a, a'') \le\operatorname{DIST}(a, a')\}|

Definition 2.2 (Matching Rank). The rank of a potential matching ⟨a,a′⟩\langle a, a' \rangle is given by the list of all matches ordered by distance:

MRANK⁡(a,a′)=∣{⟨a′′,a′′′⟩∣DIST⁡(a′′,a′′′)≤DIST⁡(a,a′)}∣\operatorname{MRANK}(a, a') = |\{\langle a'', a'''\rangle\mid\operatorname{DIST}(a'', a''') \le\operatorname{DIST}(a, a')\}|

We refer to the set of all matchings of rank kk or below as the “top-kk” candidate matching.

Direct Schema Matching. Early methods relied on surface-level similarities like lexical overlap or distributional similarity using heuristics that limited their ability to capture deeper semantic relationships. An earlier work [21] framed schema matching as a machine learning classification problem using metadata and data features, showing that learned models outperform traditional heuristic methods. Now, some more recent works capture deeper semantics using embeddings [26] and pretrained language models [38, 56, 72]. In this paper, we evaluate the distance functions of the following schema matching techniques.

Cupid [41] represents schemata as trees to capture the hierarchy of elements (relations, attributes, etc.). Similarity between elements is computed as a weighted combination of linguistic (name) and structural (contextual) similarities.

Similarity Flooding [48] transforms schemas into directed graphs and merges them into a propagation graph. Similarities propagate iteratively between connected elements until convergence.

COMA/COMA++ [4, 20] models schemas as directed acyclic graphs and combine multiple matchers (schema-based and instance-based).

Distribution-Based Matching [75] computes similarities between columns based on the distribution of their data values. It clusters relational attributes using the Earth Mover’s Distance and outputs disjoint clusters of related attributes.

Jaccard-Levenshtein Matcher- a simple baseline that computes pairwise column similarity using Jaccard similarity, treating values as identical if their Levenshtein distance falls below a threshold.

Ontology Based Matchers. Ontology-based schema matching algorithms like SemProp [26] rely on a having domain-specific ontology. To our knowledge, a low level ontology for election concepts or survey variables in general is not available. Many attributes in LoStBench correspond to literal responses, rather than the ontology classes or instances used in general ontologies like DBpedia [3] and YAGO [60]. Conversely, existing survey specific ontologies like the survey ontology [62] are intended as frameworks into which an ontology might be built. As a result, ontology-based methods such as SemProp are not directly applicable.

Schema Matching with Re-ranking. With the emergence of large language models (LLMs), several studies have explored prompt-based schema matching, highlighting the sensitivity of performance to the amount and structure of context provided [55]. LLM-based schema matching techniques adopt multi-stage pipelines [13, 38, 56] to optimize LLM usage; for example, Construm [13] enhances decision quality by injecting structure-aware context into candidate selection or re-ranking under constrained budgets. In this work, we evaluate the re-ranking ability of two such systems:

Magneto [38] leverages both small and large language models. It first retrieves candidate matches using a self-supervised, fine-tuned small LM (SLM), then re-ranks them with an LLM.

DDM [56] uses sentence embeddings of prose questions from pretrained language models and then performs approximate nearest neighbor search to retrieve ranked candidate lists. It is our own previous work and the version we evaluated optionally re-ranks attributes using an LLM.

Table Enrichment. Attribute alignment also arises in the context of table enrichment problems like dataset discovery [17, 25], where the goal is to identify tables that are unionable (adding tuples) or joinable (adding attributes) with respect to a query table. That is, given some query schema SQS^{Q} and a repository of schemas S1,…,SNS_{1}, \ldots, S_{N}, the goal is to select the kk most "similar" schemas. In contrast to schema matching, the distance (similarity) measure is defined at the level of schemas, rather than individual attributes. However, a common approach is to define the distance between the two schemas in terms of a potential schema matching. For example, Nargesian et al. [51] propose a measure (c-alignments) in which the distance between two schemas is computed by proposing a schema matching between them, and then scoring the matching based on a distance measure. That is, given a computed matching M\mathcal{M} for schemas Si,SjS_i, S_j, we can define the similarity as:

sim⁡(i,j)=∑⟨a,a′⟩∈Mi,j(1−dist⁡(a,a′))(1)\operatorname{sim}(i,j) = \sum_{\langle a,a'\rangle\in\mathcal{M}_{i,j}} (1-\operatorname{dist}(a,a')) \tag*{(1)}

Crucially, even though the objective of table enrichment systems is inter-schema similarity, they are nevertheless frequently implemented in terms of inter-attribute distance measures, and can be adapted for use in schema matching. In this paper, we evaluate the following table unionability systems:

SANTOS [32] performs table union search using semantic relationships between column pairs, beyond traditional schema or value-based similarity. It leverages external knowledge bases and synthesized knowledge derived from data lakes to find matchings.

Starmie [24] formulates table union search as a representation learning problem by training column encoders using contrastive learning over table data. It computes unionability via cosine similarity between learned column embeddings and employs efficient indexing structures such as HNSW for fast retrieval.

D3L [7] approaches table union search by extracting name-, value-, format-, embedding-, and distribution-based dataset features and mapping them into a uniform distance space using hash-based indexing. Similarity between tables is computed based on distances between these feature representations, enabling efficient retrieval of related tables.

MetaTUS [47] performs table union search using only metadata, avoiding reliance on raw data values to address privacy constraints. It models unionability via schema-level and descriptive metadata features, enabling integration under restricted data access while still aligning with FAIR principles [46].

Other works we were not able to obtain for evaluation include TACTUS [66], which introduces a table-centric approach that first retrieves candidates using table-level embeddings and subsequently refines matches at the column level, improving both efficiency and ranking quality. Complementary to union search, join discovery methods [23, 76, 77] focus on extending tables horizontally. Evaluation of table union search methods is supported by diverse benchmarks, including LLM generated datasets [53], human-annotated corpora [15], and synthetic benchmarks [19, 32, 51, 63] constructed by decomposing large seed tables.

Other Related Work

Table Representation. Closely related to the above two tasks is table representation learning, which underpins a wide range of tabular data applications. Prior work explores different objectives, including table understanding (TURL [18]), question answering (TABBIE [31]), and semantic type detection [29, 65, 74]. Notably, several methods for schema matching and table union search [24, 33, 47, 72] build upon similar representation techniques. However, learning meaningful representations from data instances remains challenging, particularly on high-entropy numerical data [34]. To address this, Pythagoras [35] proposes a graph based formulation of tables combined with graph neural networks, demonstrating improved performance in capturing semantics.

Merging Data Models. Most schema matching approaches focus on pairwise alignment; however, real-world data integration scenarios often involve multiple schemas, leading to the schema matching network problem [30, 40, 64]. Rodrigues et. al [61] addresses this problem of schema matching in multi-schema environments by proposing machine learning based technique that leverage network constraints and bootstrapped training data. In such settings, curators must identify correspondences across many schemas simultaneously, a task with combinatorial complexity. This challenge is especially pronounced in longitudinal studies, where multiple schema versions must be aligned and consolidated into a unified representation. So, beyond identifying correspondences, an equally important task is schema merging: given multiple schemas and their mappings, the goal is to construct a duplicate free and conflict resolved global model. Early work by Buneman [10] provides a theoretical foundation by modeling schemas as graphs and defining merge operations based on information ordering. Pottinger and Bernstein [57] propose a general framework for resolving duplicates and conflicts during merging. Other approaches focus on practical interoperability across heterogeneous models; for example, Quix et al. [58] introduce a metamodel based solution that transforms schemas into a metamodel, followed by human assisted mapping using real world semantics prior to merging.

When extending beyond two schemas, the merging process becomes more complex due to ordering effects and accumulated inconsistencies. To address this, Boubakir and Chaoui [8] propose a similarity-driven, iterative pairwise merging strategy that prioritizes merging closely related schemas to improve overall quality.

Integrating Longitudinal Datasets. Prior work on integrating longitudinal datasets remains relatively limited. Many studies rely on manual curation supplemented with rule-based semi-automated techniques for interoperability across datasets [11, 49, 50]. For example, in a panel study on poverty [11], person-level record linkage is performed using deterministic, rule-based matching strategies. In healthcare domains, integration efforts frequently focus on aligning schemas to predefined standards or ontologies [45, 50], or transforming data instances into structured mathematical representations [49]. Additionally, some works address construct level alignment by fusing heterogeneous measurement scales across different datasets to enable unified modeling [44].

In a related direction, [37] integrates cross-sectional and longitudinal data by generating pseudo time-series from the former and calibrating them using a small number of longitudinal samples to support disease progression modeling. Other works, such as [39], integrate structured and unstructured electronic health record data by linking patient records across modalities using deep learning techniques.

Recent approaches have also explored the use of pre-trained language models to support longitudinal schema alignment. These methods leverage the semantic content of survey instruments or variable descriptions, rather than relying on data values, to identify candidate correspondences [45, 56]. Typically, similarity-based matching or clustering is used to propose alignments, which are subsequently validated and refined by domain experts to ensure semantic consistency.

Finally, some approaches do not directly integrate datasets, but instead combine insights derived from longitudinal data. These include ensemble frameworks that combine predictions from multiple longitudinal modalities using models such as LSTMs [67], as well as methods that incorporate features or latent representations from longitudinal data into analysis models for improved predictions [52]. Related efforts also explore knowledge-level integration by linking survey questionnaire concepts through knowledge graphs, enabling adaptive systems that dynamically tailor questions based on user responses [54].

Attribute Alignment for Longitudinal Studies

In contrast to basic schema matching settings, where the focus is on integrating pairs of schemas, longitudinal datasets consist of sequences of schemas S1,…,SNS_1, \ldots, S_N. Ultimately the goal is the same: constructing a single target schema STS^T by discovering inter-attribute mappings across the entire schema. We denote the set of mappings between schemas SiS_i and SjS_j as Mi,j\mathcal{M}_{i,j}.

Pairs of successive schemas SiS_i, Si+1S_{i+1} frequently have significant overlap. That is, for many distance measures, the distance between many attribute pairs in successive schemas is negligible (i.e., given ⟨a,a′⟩∈Mi,i+1\langle a, a' \rangle\in\mathcal{M}_{i,i+1}, w.h.p. DIST⁡(a,a′)≈0\operatorname{DIST}(a,a') \approx0). We leverage this fact through an iterative integration strategy, where the target schema is incrementally developed by rolling up one schema at a time, in sequence. This strategy is summarized in Algorithm 1.

Algorithm 1 IncrementalIntegration(S1,…,SNS_1, \ldots, S_N)

Require: S1,…,SNS_1, \ldots, S_N: A sequence of schemas

Ensure: STS^T: The target schema, H1,…,HN\mathcal{H}_1, \ldots, \mathcal{H}_N: Mappings to STS^T

ST←S1S^T \leftarrow S_1; define H1(a)=a\mathcal{H}_1(a)=a

for all i←2…Ni \leftarrow2 \ldots N do

compute Mi\mathcal{M}_i between STS^T and SiS_i

ST←ST∪(Si−{a∣⟨a,_⟩∈Mi})S^T \leftarrow S^T \cup(S_i - \{a \mid\langle a, \_ \rangle\in\mathcal{M}_i\}) ▹\triangleright Add unmatched attrs.

define Hi(a)={a′if ⟨a′,a⟩∈Miaotherwise\mathcal{H}_i(a)=\begin{cases} a' & \text{if } \langle a',a\rangle\in\mathcal{M}_i \\ a & \text{otherwise} \end{cases}

end for

We note several subtle details about this algorithm. First, when computing each matching Mi\mathcal{M}_i, we have access to metadata for every preceding attribute linked into STS^T; thus exact matches can be constructed immediately. Second, in practice, matching heuristics are imprecise, and so human direction is frequently critical. We note that the process of constructing matchings is an ideal point for human intervention, as it closely mirrors the iterative nature of incremental data releases for long running projects like ANES.

Wide Table Support

Survey instances in our representative use case of ANES are quite wide (discussed in detail in § 4.1), with 100 to 2200 attributes per version. Such large schemas are common to many intensive surveys of this type, and so to be useful for longitudinal studies, schema matching tools must support wide schemas. Unfortunately, not all of them presently scale to schemas with thousands of attributes. For example, MagnetoGPT [38] is limited by its LLM-based re-ranking step, where long candidate lists (that too containing long prose questions) quickly exceed prompt limits, or degrade performance due to context loss. Even metadata-based methods like Cupid [41] do not scale well because it performs exhaustive pair-wise comparisons and repeatedly refines matches using structural relationships in the schema. Similarly, SANTOS [32] functional dependency computation step can take in excess of 24 hours for as few as 200 attributes.

Fortunately, large surveys like ANES follow a hierarchical structure that gives us a way to scale attribute-limited alignment schemes. ANES attributes are clustered into groupings of thematically related questions (denoted CATEGORY(aa)) that correlate with schema matchings. When aligning two schemas, we block [5] categories through a technique similar to table union search [51]. Let the category set of schema SiS_i be Ci=⋃a∈SiCATEGORY⁡(a)C_i = \bigcup_{a \in S_i} \operatorname{CATEGORY}(a) For each category c∈Cic \in C_i, we vertically partition SiS_i into sub-schemas:

Si,c={ a∣a∈Si∧CATEGORY⁡(a)=c }S_{i,c} = \{\, a \mid a \in S_i \land\operatorname{CATEGORY}(a) = c \,\}

Recall that we can compute the similarity between pairs of schemas Si,c,Sj,c′S_{i,c}, S_{j,c'} with Equation (1) by performing a schema match over the two schemas. For each Si,cS_{i,c}, we find the c′∈Cjc' \in C_j that maximizes sim⁡(Si,c,Sj,c′)\operatorname{sim}(S_{i,c}, S_{j,c'}) and take the union of all such matchings. This approach is quadratic in the number of categories, but like blocking in record linkage [5], avoids being quadratic in the attributes.

LoStBench — Case Study

When selecting a case study for our experiments, we wanted a long-running longitudinal study with publicly available data and a well-curated ground-truth schema. The American National Election Studies (ANES) Survey [1] is a long-running program of academically administered national surveys conducted before and after U.S. presidential elections, and fits these criteria precisely. ANES captures a continuous time series of data dating back to 1948, and is widely regarded as a gold standard in political science research. Questions, answers, and metadata in ANES are all representative of the use cases outlined in § 1.1, but it is distinct, and so better allows us to assert the generality of our results. Crucially, the ANES dataset is released alongside a well documented, hand-curated ground-truth target schema that explicitly declares how each source attribute maps into it.4 The presence of an exact ground truth makes ANES an ideal candidate for LoStBench.

The ANES Dataset

The ANES dataset contains 32 biennial time-series studies spanning the years 1948 to 2024, and instance-level data summaries are released alongside metadata, data dictionaries, and other documentation.5 Following the notation introduced in § 2, the survey explicitly provides (i) a ground truth schema STS^T, (ii) source datasets with schemas S1,…,S32S_1, \ldots, S_{32}, and (iii) mappings Hi:Si→STH_i : S_i \to S^T. Each dataset consists of responses to questions (attributes) a∈Sia \in S_i, and for each question ANES provides several forms of metadata that can be used for attribute alignment. For each of the attributes (i.e., ∣ST∪(⋃iSi)∣\lvert S^T \cup(\bigcup_i S_i)\rvert), we obtained:

LABEL⁡(a)\operatorname{LABEL}(a) — a short, mnemonic, semantically meaningful identifier;

PROSE⁡(a)\operatorname{PROSE}(a) — the prose text of the question asked;

DOM⁡(a)\operatorname{DOM}(a) — the domain of responses to the question or their encodings (e.g. 1–5 for likert scale responses);

CATEGORY⁡(a)\operatorname{CATEGORY}(a) — a thematic group of questions to which the question belongs. Notably, categories vary across years.

We emphasize that these classes of metadata are representative of the types of metadata found across all of our motivating use cases.

Attribute mappings and several forms of metadata were not available in machine-readable form, but rather presented in codebooks.

As the codebooks for each dataset followed largely consistent formatting patterns, and so we were able to script the data extraction process. Extracted mappings and metadata were cross-checked and discrepancies manually resolved. Using this process, we obtained source variable metadata for 30 survey years: (i) We omitted the 1996 release due to missing category information, and (ii) we excluded the 2024 release as it was incomplete at the time of our analysis. We obtained 10,320 source-to-target mappings (i.e., the range of all HiH_i) across the 30 years.

Contrast with Other Benchmark Datasets

Table 1 contrasts the ANES dataset with benchmark datasets frequently used to evaluate non-longitudinal attribute alignment systems. In addition to baseline statistics, we report the Szymkiewicz–Simpson coefficient6 to quantify overlaps between pairs of tables (Si,Sj)(S_i, S_j). ANES has 30 schema versions, so we consider 29 pairs of temporally adjacent versions as table pairs (Si,Si+1)(S_i, S_{i+1}). For attribute names, overlap is computed over the sets {LABEL⁡(a)∣a∈Si}\{\operatorname{LABEL}(a) \mid a \in S_i\} and {LABEL⁡(a)∣a∈Sj}\{\operatorname{LABEL}(a) \mid a \in S_j\}, and likewise for data instances using the corresponding domains DOM⁡(⋅)\operatorname{DOM}(\cdot). A value of 1 indicates identical schemas (resp., data domains), while a value of 0 indicates no overlap; Lower values indicate greater diversity in the dataset. For value overlap, we ignore numeric-valued columns.

The number of attributes (i.e., questions) in ANES is significantly higher than other benchmarks, with even the smallest schema over four times the median table size in the other benchmarks, and the largest almost three times as large. Name overlap is low, in fact among the lowest in the lot. Value overlap, conversely, is moderately higher (around the median across benchmarks) due to question patterns with standardized response categories such as “YES”, “NO”, “DON’T KNOW”, or “AGREE”.

Dataset#Table Pairs#Cols#RowsMatch TypeName Overlap MeanName Overlap VarianceValue Overlap MeanValue Overlap Variance
GDC1016–73693–4.5kHuman-Curated0.0490.0090.0770.004
Magellan73–70.9k–131kHuman-Curated1.000.3150.039
WikiData413–205.4k–10.8kHuman-Curated0.0760.0030.5810.097
Open Data17926–5111.6k–23kFabricated0.0950.0630.3090.078
ChEMBL18012–237.5k–15kFabricated0.100.0670.1480.102
TPC-DI17611–227.5k–15kFabricated0.1040.0700.2110.104
ANES2967–2.2k0.6k–8.3kHuman-Curated0.0740.0110.2550.011

Table 1. Comparative Summary of ANES schemas with other popular schema matching benchmark datasets

LoStBench — Methodology

Our primary goal in this paper is to understand the degree to which existing techniques for schema matching and adjacent problems can be re-used for longitudinal data integration tasks. By extension, in situations where they are not sufficient, we would like to understand why, and to propose fixes; Indeed, we have already done so in § 3.1. This give us our overarching research question, which we address through three more targeted questions.

RQ 1. How effective are existing attribute alignment schemes on longitudinal data integration tasks?

Dataset#Table Pairs#Cols#RowsMatch TypeName OverlapValue Overlap
MeanVarianceMeanVariance
GDC1016–73693–4.5kHuman-Curated0.0490.0090.0770.004
Magellan73–70.9k–131kHuman-Curated1.000.3150.039
WikiData413–205.4k–10.8kHuman-Curated0.0760.0030.5810.097
Open Data17926–5111.6k–23kFabricated0.0950.0630.3090.078
ChEMBL18012–237.5k–15kFabricated0.100.0670.1480.102
TPC-DI17611–227.5k–15kFabricated0.1040.0700.2110.104
ANES2967–2.2k0.6k–8.3kHuman-Curated0.0740.0110.2550.011

Table 1 (PDF p. 6). Comparative Summary of ANES schemas with other popular schema matching benchmark datasets

Pairwise Alignment

First, we isolate differences that arise from the nature of the data itself: We want to know how effective the existing schema matching algorithms are at working with ANES schemas and data. As we already noted, existing alignment schemes struggle with the wide schemas of ANES, but there are a range of other factors that we want to consider. For example, the prose questions (PROSE⁡(⋅)\operatorname{PROSE}(\cdot)) carry a much stronger signal about alignment than attribute labels (LABEL⁡(⋅)\operatorname{LABEL}(\cdot)), but we want to know whether existing schemes are capable of using this signal.

RQ 2. How effective are existing attribute alignment schemes when used to align pairs of survey revisions?

We address this question by evaluating each form of attribute alignment on its ability to align pairs of schemas SiS_i and SjS_j drawn from the ANES data. The alignment technique’s success is evaluated through three measures: (i) Recall@GroundTruth (R@GT), (ii) Mean Reciprocal Rank (MRR), and (iii) Rank For Ideal Recall (RIR). The first measure models the technique in an idealized setting, measuring its ability to act without supervision. An ideal ranking of R@GT = 1 suggests that the algorithm may be used with minimal supervision.

Definition 5.1 (R@GT). Let MGM^{G} be the ground-truth matching for a pair of schemas SiS_i, SjS_j. We are given a top-∣MG∣|M^{G}| matching MM (Theorem 2.2); We define the Recall@GroundTruth for MM as the fraction of it that is correct:

R@GT⁡=∣M∩MG∣∣MG∣\operatorname{R@GT} = \frac{|M \cap M^{G}|}{|M^{G}|}

Unlike R@GT, which considers a global ranking of all candidate matches across variables, a mean reciprocal rank (MRR) evaluates the rankings individually for each source variable. It is a standard metric commonly used in evaluating ranked retrieval systems, such as search engines and question-answering systems [43].

Definition 5.2 (MRR). Let MGM^{G} be the ground-truth matching for schemas SiS_i, SjS_j and recall that ARANK⁡a(a′)\operatorname{ARANK}_{a}(a^{\prime}) is the rank of a′a^{\prime} with respect to aa. Define the score of a∈Sia \in S_i to be the minimal rank of a correct attribute matching for aa:

score⁡(a)=min⁡⟨a,a′⟩∈MGARANK⁡a(a′)\operatorname{score}(a) = \min_{\langle a,a^{\prime}\rangle\in M^{G}} \operatorname{ARANK}_{a}(a^{\prime})

Let μ\mu be the set of attributes with at least one correct match: μ={a∣⟨a,_⟩∈MG}\mu= \{a \mid\langle a,\_ \rangle\in M^{G}\}. The mean reciprocal rank is the average of the reciprocal ranks for all matching attributes:

MRR⁡=1∣μ∣∑a∈μ1score⁡(a)\operatorname{MRR} = \frac{1}{|\mu|} \sum_{a \in\mu} \frac{1}{\operatorname{score}(a)}

Intuitively, MRR captures how quickly the first relevant match appears when examining the ranked list of candidate matches. A higher MRR indicates that the algorithm ranks correct matches near the top of the list, allowing users to more easily identify them in human-in-the-loop settings. But MRR is only informative and helpful until we are looking for 1 relevant match. If we are matching more than two schemas, or a variable has multiple matches, then in such cases MRR doesn’t quantify how far along we need to look for in a ranked list for all relevant matches. The further down the list the correct match appears, the more work the manual curator must perform. To quantify this, we propose a new measure: Rank For Ideal Recall.

Definition 5.3 (RIR). Let MGM^{G} be the ground-truth matching for schemas SiS_i, SjS_j and recall that ARANK⁡a(a′)\operatorname{ARANK}_{a}(a^{\prime}) is the rank of a′a^{\prime} with respect to aa. Given a∈Sia \in S_i, the rank for ideal recall of aa in MGM^{G} is:

RIR⁡(a)=max⁡⟨a,a′⟩∈MGARANK⁡a(a′)\operatorname{RIR}(a) = \max_{\langle a,a^{\prime}\rangle\in M^{G}} \operatorname{ARANK}_{a}(a^{\prime})

The RIR for an alignment is the sum over matched attributes: RIR = ∑a∈μRIR⁡(a)\sum_{a \in\mu} \operatorname{RIR}(a) where μ\mu is defined as in Definition 5.2.

Category Alignment

As outlined in Section 3.1, some existing alignment techniques lack support for ANES’s wide schemas. To address this limitation, we leverage the hierarchical structure of survey instruments like ANES to restrict schema matching to within matched categories. However, this optimization is predicated on the attribute alignment technique being able to correctly match categories. We next ask whether this is the case.

RQ 3. Can attribute alignment schemes effectively match categories across schema revisions?

We answer this question by re-interpreting LoStBench as a table unionability benchmark, partitioning schemas into individual per-category (i.e., CATEGORY⁡(⋅)\operatorname{CATEGORY}(\cdot)) tables, and drawing a sample as described in Section 6. Table 2 contrasts LoStBench with other table unionability benchmarks. Boutalet et al. [9] identified three key limitations in such benchmarks: semantic simplicity, ground-truth inconsistencies, and excessive overlap. As previously noted, semantic simplicity is a limitation of the longitudinal study domain. However, the survey questions in ANES cover a wide range of general social science variables, including personal, economic, educational, and political topics, providing meaningful diversity for analysis. Ground-truth inconsistencies are expected to be minimal, as the dataset is derived from a source with over 10,000 citations, maintained manually for more than 75 years, and periodically corrected. In terms of overlaps, LoStBench exhibits the lowest name overlap and the second-lowest value overlap among the benchmarks.

The LoStBench table unionability workload is comparable in shape and cardinality to other table unionability workloads, but differs significantly in terms of its data. Like many social science surveys, participants may elect to omit answers and almost a third of the data is null. By contrast, even the most sparse table unionability benchmark dataset has only a quarter null values. We also note that in contrast to other table unionability benchmarks, where null values indicate the absence of semantic content, an individual’s choice to skip an answer is often semantically meaningful to statisticians using datasets such as ANES. A second distinguishing feature of ANES is that responses are frequently categorical, and rarely (<5%) numerical in contrast to other benchmark datasets. We note that comparatively few table unionability schemes support numerical data, so this is a positive feature for existing approaches.

When evaluating table unionability, we focus on three measures.

BenchmarkTypeTable Category#TablesAvg ShapeNull(%)Numeric(%)Non-Numeric(%)Size (MB)Name OverlapValue Overlap
UGEN V2LLM-GeneratedCandidates100019 × 145.8717.1782.83∼70.2570.422
Query50108 × 144.4512.3387.67
SANTOSFabricatedCandidates5006922 × 127.6125.6474.36∼4400.7810.674
Query5021402 × 135.8823.9076.1
Real TablesFabricatedCandidates110737719 × 1122.6026.0373.97∼11,5000.8370.601
Query8012918 × 1316.9026.3573.65
TUS LargeFabricatedCandidates50431915 × 1112.538.6791.33∼1,5900.6800.509
Query42962132 × 1814.877.8192.19
Lakebench-OpenDataFabricated & Human-CuratedCandidates4832112,360 × 213.4444.9355.07∼80,0000.6130.453
Query313879,311 × 172.9053.8746.13
PYLONHuman-CuratedCandidates1746115 × 10-41.2658.74∼220.1550.252
Query12490 × 7-24.3275.68
ANES/LoStBenchHuman-CuratedCandidates13862417 × 932.053.5596.45∼3850.0880.281
Query1152049 × 928.364.4095.60

Table 2. Table Union Search Benchmarks Comparison

Definition 5.4 (P@K). Precision at rank KK measures the fraction of relevant items among the top-kk retrieved results.

Precision⁡@K=1K∑k=1Krel⁡(k)\operatorname{Precision}@K = \frac{1}{K}\sum_{k=1}^{K}\operatorname{rel}(k)

where rel⁡(k)∈{0,1}\operatorname{rel}(k) \in\{0,1\} indicates whether the item at rank kk is relevant (i.e, in the ground truth).

Definition 5.5 (AP@K). Average Precision at KK computes the average of Precision@k values at positions where relevant items occur, thereby rewarding methods that retrieve relevant items earlier and penalizing for errors in the early ranks.

AP⁡@K=1min⁡(N,K)∑k=1KPrecision⁡@k⋅rel⁡(k)\operatorname{AP}@K = \frac{1}{\min(N,K)}\sum_{k=1}^{K}\operatorname{Precision}@k \cdot\operatorname{rel}(k)

where NN is the total number of relevant items.

Definition 5.6 (MAP@K). Mean Average Precision at KK is the mean of AP@K over all queries.

MAP⁡@K=1∣Q∣∑q∈QAP⁡@K(q)\operatorname{MAP}@K = \frac{1}{|Q|}\sum_{q \in Q}\operatorname{AP}@K(q)

where QQ is the set of queries and AP@K(q) is the Average Precision at K for query q.

Longitudinal Alignment

In Section 3, we propose a strategy for generalizing pairwise schema matching to matching a sequence of schemas. This approach comes with a trade-off: More steps are required, but each step is significantly more targeted — the curator has fewer potential alignments to review (and discard) in each step. Our final question considers the utility of this approach.

RQ 4. Does incremental alignment reduce the effort required to integrate a longitudinal study.

Rank for Ideal Recall (RIR) (Definition 5.3) measures the number of matchings that the user must consider in order to fully integrate a dataset, and so we use this measure as a proxy for effort.

Experiments

Our overarching goal (RQ 1) is to understand how effective existing schema matching algorithms can be when applied to the iterative evolution of a longitudinal study. We now address this goal through experiments that (i) use pairwise schema matching to isolate domain-specific effects of ANES on schema matching algorithms (RQ 2, addressed in Section 6.1); (ii) confirm the value of incremental matching (RQ 4, addressed in Section 6.2); and (iii) examine the efficacy of the blocking optimization from Section 3.1 (RQ 3, addressed in Section 6.3).

Experimental Setup. Section 2 reviews and motivates the choice of systems being evaluated, and section 4 documents how we adapted these systems to the setting of longitudinal data integration, and to wide attribute schemas. Most schema matching systems are intended for use with attribute identifiers LABEL⁡(a)\operatorname{LABEL}(a), but because of the additional semantic value present in PROSE⁡(a)\operatorname{PROSE}(a), we attempted to use the latter as an identifier with each system. We report these results if they were better.

For COMA/COMA++, Cupid, Sim Flooding, Distr-based, and Jaccard-Lev, we used the publicly available implementation from [33]. We ran these algorithms with all input parameter combinations suggested in the original paper, and report results from the optimal combination of settings. The results of Cupid [41] were significantly worse than other approaches, and so are omitted from our reporting. For Magneto we used it’s publicly available implementation, whereas DDM is our own previous work. The embedding model used for DDM was paraphrase-MiniLM-L6-V2. We evaluated Magneto using its zero-shot configurations (Magneto-zs-bp and Magneto-zs-llm) and did not perform fine-tuning. In the general schema alignment case, we do not expect labeled data for fine-tuning to be readily available, requiring practitioners to rely on off-the-shelf models. Furthermore, attribute semantics are largely conveyed through general-purpose natural language in question text, which aligns well with the pretraining distribution of modern language models like mpnet used by Magneto. We also experimented with each of Magneto’s serialization strategies (default, verbose, and repeat), and got slightly better results for verbose serialization compared to the others; we report these results only. For MagnetoGPT and DDM_LLM we used gpt-4o-mini as LLM re-ranker. We used the publicly available implementations of SANTOS, Starmie, D3l, and MetaTUS. In the case of SANTOS experiment, it was additionally necessary to identify intent columns, a process they performed manually in their own evaluation. In LoStBench, we generated intent columns by using the attribute with the highest number of matched attributes in the ground truth. We pretrained Starmie on LoStBench by exploring a subset of the combinations of hyperparameter space, varying augmentation operators and sampling methods while keeping other parameters at their recommended values as mentioned in the documentation. Then we performed the search with naive, HNSW, and LSH techniques on different threshold and kk-values on recommended hyperparameters.

The preprocessing and index generation for SANTOS were carried out on a machine with 1 TB of memory to accommodate the in memory processing of raw DBpedia knowledge base, consisting of single files of over 100 GB each. All other workloads for the methods described above were executed on a machine running Pop!_OS 22.04 LTS (Ubuntu 22.04), equipped with an AMD Ryzen 7 8700G processor with 8 physical cores and 16 threads, and 64 GB of system memory. GPU-based methods utilized the GPU for computation, except for the scalability experiment, which was run on the CPU to maintain consistency with the other methods.

Experiment 1: Schema Matching

Our first experiment measures the effectiveness of existing schema matching algorithms on pairwise matching. Recall that ANES attributes are organized into groups of related questions called categories. We model the wide-schema optimization from § 3.1, by selecting matched pairs of categories from adjacent survey iterations and evaluating the efficacy of the schema matching tool on the pair. For constructing the test set for schema matching, we aimed for a reasonable pair count because we were dealing with methods making api calls to LLMs, that could potentially take a minute to run for each pair. Additionally, we aimed to maximize schema diversity. To this end, we ensured that no category appeared more than twice across the sampled test cases. We performed stratified sampling based on the number of linked variables in each category pair. Since the distribution was skewed, most pairs had only two linked variables with progressively fewer pairs at higher link counts. With this setup, preliminary experiments resulted in the discovery of two significant limitations of existing schema matching systems that we discuss below.

Missing Values. Consider the following block of questions:

D5. (OCCUPATION OF HEAD OF HOUSEHOLD) WHAT IS (YOUR/THE HEAD OF YOUR HOUSEHOLD'S) OCCUPATION? I MEAN, WHAT KIND OF WORK (DO YOU/DOES HE) DO?

D6. (DO YOU/DOES HE) DO ANY OTHER KIND OF WORK BESIDES FARMING—FOR PAY, I MEAN? THE FOLLOWING THREE QUESTIONS ARE FOR FARMERS ONLY:

D7. (DO YOU/DOES HE) OWN (YOUR/HIS) FARM, MANAGE IT, RENT IT, OR WHAT?

D8. HOW MUCH LAND (DO YOU/DOES HE) FARM?

D9. WHAT KIND OF FARMING (DO YOU/DOES HE) DO HERE MOSTLY?

In years where no farmers participated in a given survey, response to the questions (D7-D9) are entirely empty. This happens quite frequently — of the 2,056 categories seen in 30 iterations of ANES datasets, 670 have at least one column completely empty. Cases like these convey no semantic content for instance-based models, making them effectively useless. For further experiments, we restricted ourselves to the remaining 1,386 categories with non-empty instances.

Redundancy. Redundancy also plays a significant role in survey design, with repeated or rephrased questions used to detect bias or correct errors [14]. Many schema matching and table unionability algorithms rely on heuristics that reduce the search space by explicitly excluding duplicate matches (e.g., c-alignment [51]). Such heuristics are not appropriate for survey data. For further experiments, we eliminated many-to-many and one-to-many attribute pairings.

Considering the missing values and redundancies, we refined our sampling process to omit parts of the problem domain that existing systems are incapable of addressing. From this point we applied weighted sampling- more weights to pairs with higher no. of links- to downsample to 600 category pairs. To further balance the distribution, we randomly removed 150 pairs containing only two linked variables, resulting in a final set of 450 category pairs for the test set, twice the size of any other benchmark that appears in Table 1.

LLMs and Question Identifiers. Magneto leverages both variable names and column values for its SLM and LLM components to infer column semantics, and as noted above, we use the question prose (PROSE(aa)) in lieu of the attribute name where it is beneficial to do so. We identified two main issues with their original llm prompt layout when using PROSE(aa) as the attribute identifier. While the results for Magneto-zs-bp were good, the performance of the Magneto LLM re-ranker was nearly five times worse than the SLM component. For the original LLM prompt employed in Magneto-zs-llm, the LLM frequently failed to produce output that could be successfully parsed, and in several cases the parser had to abort for a variable after five retries. One problem was that the LLM was unable to consistently reproduce the long prose attribute names. For instance, one question was titled “Q.J6. What about the next 12 months? Do you...” When identifying to this question in the re-ranking step, the LLM might inject an additional space between the ’?’ and the ’D’. Though interpretable to a human, this alteration disrupted Magneto’s parser. In other cases, the same attribute captures the responses to multiple linked sub-questions under a single merged scale (e.g. “Q.J5a.” and “Q.J5b.”). Because the prose for these questions was concatenated, the LLM’s output would treat each sub-question as an independent question, leading to more parser disruption.

Comparison of Schema Matching Methods

Figure 3. Comparison of Schema Matching Methods

Figure 2 illustrates a representative schema matching performed using a variety of systems, both in the raw form (No LLM) and with an LLM based re-ranking step. Notably, using prose⁡(a)\operatorname{prose}(a) in place of attribute name, the LLM re-ranking step decreased accuracy, resulting in an accuracy almost five times worse. Reverting to label⁡(a)\operatorname{label}(a) as an attribute name (Lbl- Both) prevented LLM parser failures, and in turn prevented re-ranking errors. However, this came at the cost of a substantial reduction in accuracy of the pre-LLM step, leading to an overall reduction in accuracy. We also explored a third, hybrid approach (S-Q, L-lbl) by modifying Magneto to use prose⁡(a)\operatorname{prose}(a) in the initial SLM step, and label⁡(a)\operatorname{label}(a) as a variable identifier in the LLM prompt. This preserved the SLM component’s performance but with the absent the prose signal, the re-ranking step reduces accuracy.

Figure 2. Improvement of LLM re-ranker with changes in prompts. (Note: S-Q, L-lbl means SLM-Qn, LLM-lbl)

Our final effort generates a re-ranking prompt that uses label⁡(a)\operatorname{label}(a) as an identifier, but also provides prose⁡(a)\operatorname{prose}(a) in a “Detailed meaning of the column” field within the prompt. This revision resulted in a slight improvement in accuracy from re-ranking. We explored a similar revision in the context of DDM; Leveraging its insight that context plays a role in longitudinal studies, we also provided the LLM with all the preceding variables in the schema when prompting the LLM for a given variable in the schema.

Comparative Analysis

fig:3a shows pairwise schema matching performance. The results show a clear performance stratification across the evaluated schema matching algorithms. Even with missing values omitted, instance-based methods perform poorly. ANES questions follow a set of common patterns, and so relying solely on literal data values provides limited semantic information to distinguish between differences in column meanings. In contrast, metadata-based and hybrid approaches benefit from the richer semantic content present in the associated prose questions.

fig:3b shows that the top four models in terms of Recall@GTRecall@GT and MRRMRR also exhibit the lowest RIR values, albeit at the cost of substantially increased runtime. Recall that RIR serves as a proxy for human-in-the-loop effort: Magneto’s 1.34 RIR indicates that in expectation, all correct matchings for an attribute appears within the first 1.34 predicted pairings for an attribute. By contrast, DDM, a simple approach based on word embeddings, is also able to place the correct result within the top 2 positions on average, while executing in only a tenth of a second compared to nearly 3 seconds for Magneto, or three quarters of a minute for MagnetoGPT.

Ablation Study: Frequency of LLM Hallucination and Re-Ranker Upper Bound

While the LLM re-ranker improved results in several cases, we observed that it often underperformed compared to the SLM component whose ranked output it consumes. To better understand this behavior, we conducted an ablation study to quantify how frequently the LLM re-ranker produces stochastic errors (hallucinations) relative to its SLM input and to estimate an upper bound on Recall@GT. We performed this experiment using DDM. For each test instance, we compared the Recall@GT of the LLM re-ranked output with that of the original SLM-ranked list. Whenever the LLM re-ranker produced a lower Recall@GT than the SLM, we provided feedback and prompted the LLM to regenerate the ranked list using the same input. This process was repeated until the LLM achieved a Recall@GT at least equal to that of the SLM, or until a maximum of 10 retries was reached, after which the instance was skipped. This setup allowed us to estimate both the frequency of stochastic errors and the achievable upper bound under iterative correction. We observed that in 90 out of 450 instances (20% instances), the LLM re-ranker required at least one retry, indicating a stochastic error. Under this retry-based regime, the average Recall@GT improved to 0.827, compared to 0.821 in the standard single-pass setting (as shown in Figure 2). These results suggest that while the LLM re-ranker has the potential to improve performance, it is prone to instability, and its gains are bounded even under idealized corrective feedback.

Experiment 2: Iterative Schema Matching

Our second experiment evaluates the iterative schema matching technique from § 3. Concretely, we model a setting where a user reviews and manually corrects the schema matching at each iteration MiM_i before the process proceeds. We use DDM as a representative schema matching algorithm, as it is the most accurate sub-second runtime algorithm. Concretely, we use DDM to generate a ranking, compute its RIR. We model the user manually curating the matching by replacing the matching with the correct ground-truth matching. We consider three variations on this approach: (i) Algorithm 1 with schemas integrated in temporal order (Temporal), (ii) Algorithm 1 with schemas integrated in random order (Random), and (iii) An all-at-once, NN-way merge where the user instead fully constructs one attribute from the ground-truth schema at a time. Our objective is to analyze how the merging strategy and schema order influence curator effort, as captured by the AUC (Area Under the Curve) of RIR.

Test Set for Iterative Schema Alignment. Given the set of mappings H\mathcal{H}, we construct, for each schema SiS_i, a set of schemas RiR_i from subsequent years that share correspondences with SiS_i.

Ri={Sj∣j>i∧∃ (a,a′)∈Mi,j such that a∈Si∧y∈Sj}.R_i = \{S_j \mid j > i \land\exists\,(a,a') \in M_{i,j}\ \text{such that}\ a \in S_i \land y \in S_j\}.

We then consider the set Vi=Ri∪SiV_i = R_i \cup S_i and define an undirected graph Gi=(Vi,E)G_i = (V_i,E), where

E={(Sp,Sq)∣∃ (a,a′)∈Mp,q such that a∈Sp∧a′∈Sq}.E = \{(S_p,S_q) \mid\exists\,(a,a') \in M_{p,q}\ \text{such that}\ a \in S_p \land a' \in S_q\}.

For each ii, we verify whether GiG_i is a complete graph, i.e.,

∀Sp,Sq∈Vi, Sp≠Sq ⇒ (Sp,Sq)∈E.\forall S_p,S_q \in V_i,\ S_p \ne S_q \ \Rightarrow\ (S_p,S_q) \in E.

This process yields 71 such instances of ViV_i satisfying the completeness condition, which are subsequently used as test cases in the iterative Schema Alignment experiments.

Results. Figure 4a shows that all-at-once alignment yields higher AUC than pairwise iterative alignment, indicating greater validation and correction effort for the user. This suggests that curators benefit from distributing effort across smaller, incremental corrections

Comparison of merging techniques

Figure 4. Comparison of Merging Techniques

rather than fewer large ones, the answer to our RQ 4. Notably, the CDF in Figure 4b indicates that a temporal/chronological input ordering of schemas subjected for iterative alignment requires slightly less manual effort than a random input schema ordering when it comes to longitudinal settings.

Experiment 3: Table Union Search

As above, we focus on the 1386 categories on which all approaches have a chance of being effective. Treating each category as a table, we constructed a ranked list of unionable categories, ordering them by the number of linked variables. From this set, we stratified and sampled 115 query tables (∼\sim10%) based on the number of unionable tables associated with each query.

Limitations of Ontology-Based Alignment. For MeTaTUS, we evaluated combinations of topic embedding and topic filtering across all four modes described in the original work. However, we report results only for the base mode, as the remaining variants (dtype, dbpediadbpedia, and dbpedia+dtypedbpedia+dtype) had higher runtimes without performance improvements. These variants depend on mapping variables to external ontology classes, which proves ineffective in our setting. The dtype mode uses SHERLOCK [29] to assign variables to one of 78 DBpedia classes via semantic type detection, while the dbpedia mode employs an LLM for a similar mapping. In both cases, the assigned DBpedia classes fail to capture the semantics of survey questionnaire variables. As illustrated in Figure 5, the classification of Q.37A as language or Q.38A as region do not reflect the underlying semantics of these variables.

Examples of Semantic Type Prediction by Sherlock

Figure 5. Examples of Semantic Type Prediction by Sherlock (used as a part of ‘dtype’ mode) on LoStBench Variables.

  • Q.37. WHAT WAS YOUR OCCUPATION FOUR YEARS AGO. I MEAN, WHAT KIND OF WORK WERE YOU DOING AROUND THE TIME OF THE 1948 ELECTION. classification

  • Q.37A. (IF R WAS EMPLOYED) WERE YOU WORKING FOR YOURSELF THEN, OR WERE YOU WORKING FOR SOMEONE ELSE. language

  • Q.38. (ASK ONLY IF R IS NOT HEAD OF HOUSEHOLD) WHAT KIND OF WORK WAS THE HEAD OF YOUR HOUSEHOLD DOING FOUR YEARS AGO, AROUND THE TIME OF THE 1948 ELECTION. classification

  • Q.38A. (IF HEAD WAS EMPLOYED) WAS HE WORKING FOR HIMSELF THEN, OR WAS HE WORKING FOR SOMEONE ELSE. region

  • Q.39. WHEN YOU WERE GOING TO GRADE SCHOOL OR HIGH SCHOOL, DID YOU GO TO A PAROCHIAL SCHOOL, A PRIVATE SCHOOL, OR A REGULAR PUBLIC SCHOOL. type

  • Q.40, 40A. WERE YOU OR THE HEAD OF YOUR HOUSEHOLD EVER IN THE ARMED SERVICES OF THE UNITED STATES. (IF YES) WHICH ONE OF YOU WAS THAT. notes

  • Q.40B. (IF YES TO REF.NO.234) WHAT YEARS DID THAT COVER. notes

  • FOR HEAD OF HOUSEHOLD (INCLUDING R IF HEAD) language

  • PARTY IDENTIFICATION AND STRENGTH status

Analysis. During indexing for SANTOS, entities for 1107 tables (approximately 80%) could not be resolved from the YAGO [60] knowledge base. This is expected, as many literal values in the dataset do not correspond to entities in external knowledge bases. Nevertheless, as shown in Figure 6a, SANTOS achieves the highest Mean Average Precision@K among instance-based methods. The MAP@K score reported in Figure 6a for Starmie is for naive search. The hnsw and lsh indexing search results obtained 50% and 75% worse results respectively as compared to the naive search counterpart. Among metadata-based approaches, all three methods exhibit comparable performance in terms of precision@k. However, in contrast to other benchmark datasets where SANTOS, Starmie, and MetaTUS often exceed 90% MAP@K, performance on LoStBench is notably lower. This answers RQ 3 and further suggests that existing approaches struggle to capture the semantics of longitudinal survey data, highlighting the need for improved schema alignment models tailored to such datasets.

Comparison of Effectiveness and Efficiency for TUS methods

Figure 6. Comparison of Effectiveness (Precision@K) and Efficiency (Query and Index times) for TUS methods.

In terms of runtimes, the naive(brute-force) search for metadata-based methods (MetaTUS and DDM_Naive) are substantially faster than the instance-based approach (Starmie_Naive), as expected. The LSH and HNSW based indexing variants of Starmie achieve the lowest query times among all methods; however, this speedup comes at the cost of significantly reduced effectiveness. In contrast, the HNSW-based variant of DDM not only provides a modest improvement in runtime over DDM_Naive but also preserves MAP@K performance, despite relying on an approximate search mechanism.

Conclusions

We have presented LoStBench, a data integration benchmark for longitudinal studies based on the ANES longitudinal study dataset. ANES differs from other datasets used in evaluating data integration in ways that include: (i) Resistance to instance-based schema matching algorithms; (ii) Strong attribute matching signal in the form of prose questions; (iii) Hierarchical structure; (iv) Very wide schemas; (v) Substantial redundancy; and (vi) Sequential structure in schema mappings. We have demonstrated how existing algorithms for schema matching and related problems can be adapted to a longitudinal data integration task, and evaluated a range of such algorithms applied in this new setting. Notably, we find that metadata-based techniques outperform instance-based techniques on ANES. We also find that algorithms like embedding-based matching, which can incorporate the prose text of questions, are able to outperform other schema matching techniques. Finally, we find that techniques based on heavy-weight machine learning models are inconsistent in their ability to outperform lighter-weight models, and when they do, the resulting accuracy improvement does not justify the substantial increase in iteration time.

Through this work, we hope to motivate longitudinal studies as a compelling new use case for which new schema matching techniques may be developed. LoStBench is available as a public github repository.

Although this work represents a first step into the space, many challenges remain open. For example, surveys often involve redundancy to validate results; Not only does this result in many-to-many mappings, but also potential key violations. A second, significant difference from classical schema matching is that some mappings (e.g., salary ranges) require transformation into a common frame of reference; a problem closer to data exchange. As in data exchange, data values may be correlated, but not directly comparable. For example, salary ranges from distinct years may overlap in large part, but not entirely. Social longitudinal studies specifically exacerbate this problem, as even minor changes in question phrasing, interviewer, or survey context can bias the collected data for a given survey iteration.

References

References

  1. [1]American National Election Studies. 2026. ANES Time Series Cumulative Data File [dataset and documentation]. February 5, 2026 version. www.electionstudies.org.
  2. [2]Marcelo Arenas, Pablo Barcel, Leonid Libkin, and Filip Murlak. 2014. Foundations of Data Exchange. Cambridge University Press, USA.DOI
  3. [3]Sören Auer, Christian Bizer, Georg Kobilarov, Jens Lehmann, Richard Cyganiak, and Zachary Ives. 2007. DBpedia: a nucleus for a web of open data. In Proceedings of the 6th International Semantic Web and 2nd Asian Conference on Asian Semantic Web Conference (Busan, Korea) (ISWC’07/ASWC’07). Springer-Verlag, Berlin, Heidelberg, 722–735.
  4. [4]David Aumueller, Hong-Hai Do, Sabine Massmann, and Erhard Rahm. 2005. Schema and ontology matching with COMA++. In Proceedings of the 2005 ACM SIGMOD International Conference on Management of Data (Baltimore, Maryland) (SIGMOD ’05). Association for Computing Machinery, New York, NY, USA, 906–908. https://doi.org/10.1145/1066157.1066283
  5. [5]Rohan Baxter, Peter Christen, and Tim Churches. 2003. A comparison of fast blocking methods for record linkage. In Workshop on Data Cleaning, Record Linkage and Object Consolidation. 25–27.
  6. [6]Sonia Bergamaschi, Silvana Castano, Maurizio Vincini, and Domenico Beneventano. 2001. Semantic integration of heterogeneous information sources. Data Knowl. Eng. 36, 3 (March 2001), 215–249. https://doi.org/10.1016/S0169-023X(00)00047-1
  7. [7]Alex Bogatu, Alvaro A. A. Fernandes, Norman W. Paton, and Nikolaos Konstantinou. 2020. Dataset Discovery in Data Lakes. In 2020 IEEE 36th International Conference on Data Engineering (ICDE). IEEE, 709–720. https://doi.org/10.1109/icde48307.2020.00067
  8. [8]Mohammed Boubakir and Allaoua Chaoui. 2016. A Pairwise Approach for Model Merging. In Modelling and Implementation of Complex Systems. Springer, 327–340. https://doi.org/10.1007/978-3-319-30286-8_25
  9. [9]Allaa Boutan, Susan B. Davidson, and Anthony Kosky. 1992. Theoretical Aspects of Schema Merging. In Proceedings of the 5th International Conference on Extending Database Technology (EDBT). Springer, 152–167. https://homepages.inf.ed.ac.uk/opb/papers/EDBT1992.pdf
  10. [10]Peter Bounman, Susan B. Davidson, and Anthony Kosky. 1992. Theoretical Aspects of Schema Merging. In Proceedings of the 5th International Conference on Extending Database Technology (EDBT). Springer, 152–167. https://homepages.inf.ed.ac.uk/opb/papers/EDBT1992.pdf
  11. [11]Deven Carlson, Robert Haveman, Barbara Wolfe, Kyong Wan Kang, and Hannah Miller. 2015. Creating an Integrated Longitudinal Person and Household Dataset by Linking State-based Administrative Data. Technical Report. Institute for Research on Poverty, University of Wisconsin–Madison. https://www.neighborhoodindicators.org/sites/default/files/publications/SMR%20Progress%20report%2008%2025.pdf Accessed: 2026-04-01.
  12. [12]Shobhana L. Chelliah and Willem J. de Reuse. 2011. Handbook of descriptive linguistic fieldwork. Dordrecht: Springer.
  13. [13]Houming Chen, Zhe Zhang, and H. V. Jagadish. 2026. ConStruM: A Structure-Guided LLM Framework for Context-Aware Schema Matching. arXiv:2601.20482 [cs.DB] https://arxiv.org/abs/2601.20482
  14. [14]Kuang Chen, Harr Chen, Neil Conway, Heather Dolan, Joseph M. Hellerstein, and Tapan S. Parikh. 2009. Improving data quality with dynamic forms. In ICTD. IEEE, 487.
  15. [15]Tianji Cong, Fatemeh Nargesian, and HV Jagadish. 2023. Pylon: Semantic table union search in data lakes. arXiv preprint arXiv:2301.04901 (2023).arxiv.org/abs/2301.04901
  16. [16]Hilary Daniel, Sue S. Bornstein, and Gregory C. Kane and. 2018. Addressing Social Determinants to Improve Patient Care and Promote Health Equity: An American College of Physicians Position Paper. Annals of Internal Medicine 168, 8 (April 2018), 577. https://doi.org/10.7326/M17-2441
  17. [17]Anish Das Sarma, Lujun Fang, Nitin Gupta, Alon Halevy, Hongrae Lee, Fei Wu, Reynold Xin, and Cong Yu. 2012. Finding related tables. In Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data (Scottsdale, Arizona, USA) (SIGMOD ’12). Association for Computing Machinery, New York, NY, USA, 817–828. https://doi.org/10.1145/2213836.2213962
  18. [18]Xiang Deng, Huan Sun, Alyssa Lees, You Wu, and Cong Yu. 2020. TURL: Table Understanding through Representation Learning. arXiv:2006.14806 [cs.IR] https://arxiv.org/abs/2006.14806
  19. [19]Yuhao Deng, Chengliang Chai, Lei Cao, Qin Yuan, Siyuan Chen, Yanrui Yu, Zhaoze Sun, Junyi Wang, Jiajun Li, Ziqi Cao, Kaisen Jin, Chi Zhang, Yuqing Jiang, Yuanfang Zhang, Yuping Wang, Ye Yuan, Guoren Wang, and Nan Tang. 2024. LakeBench: A Benchmark for Discovering Joinable and Unionable Tables in Data Lakes. Proc. VLDB Endow. 17, 8 (April 2024), 1925–1938. https://doi.org/10.14778/3659437.3659448
  20. [20]Hong-Hai Do and Erhard Rahm. 2002. COMA: a system for flexible combination of schema matching approaches. In Proceedings of the 28th International Conference on Very Large Data Bases (Hong Kong, China) (VLDB ’02). VLDB Endowment, 610–621.
  21. [21]AnHai Doan, Pedro Domingos, and Alon Y. Halevy. 2001. Reconciling schemas of disparate data sources: a machine-learning approach. SIGMOD Rec. 30, 2 (May 2001), 509–520. https://doi.org/10.1145/376284.375731
  22. [22]AnHai Doan, Alon Halevy, and Zachary Ives. 2012. Principles of Data Integration (1st ed.). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.
  23. [23]Yuyang Dong, Kunihiro Takeoka, Chuan Xiao, and Masafumi Oyamada. 2021. Efficient Joinable Table Discovery in Data Lakes: A High-Dimensional Similarity-Based Approach. arXiv:2010.13273 [cs.IR] https://arxiv.org/abs/2010.13273
  24. [24]Grace Fan, Jin Wang, Yuliang Li, Dan Zhang, and Renée Miller. 2023. Semantics-aware Dataset Discovery from Data Lakes with Contextualized Column-based Representation Learning. arXiv:2210.01922 [cs.DB] https://arxiv.org/abs/2210.01922
  25. [25]Raul Castro Fernandez, Ziawash Abedjan, Famien Koko, Gina Yuan, Samuel Madden, and Michael Stonebraker. 2018. Aurum: A Data Discovery System. In ICDE. 1001–1012.
  26. [26]Raul Castro Fernandez, Essam Mansour, Abdulhakim Ali Qahtan, Ahmed K. Elmagarmid, Ihab F. Ilyas, Samuel Madden, Mourad Ouzzani, Michael Stonebraker, and Nan Tang. 2018. Seeping Semantics: Linking Datasets Using Word Embeddings for Data Discovery. 2018 IEEE 34th International Conference on Data Engineering (ICDE) (2018), 989–1000. https://api.semanticscholar.org/CorpusID:4210714
  27. [27]Sandro Galea, Melissa Tracy, Katherine J. Hoggatt, Charles DiMaggio, and Adam Karpati. 2011. Estimated Deaths Attributable to Social Factors in the United States. American Journal of Public Health 101, 8 (Aug. 2011), 1456–1465. https://doi.org/10.2105/ajph.2010.300086
  28. [28]Jeff Good. 2022. The scope of linguistic data. In The open handbook of linguistic data management, Andrea L. Berez-Kroeker, Bradley McDonnell, Eve Koller, and Lauren B. Collister (Eds.). Cambridge, MA: MIT Press, 27–47.DOI
  29. [29]Madelon Hulsebos, Kevin Hu, Michiel Bakker, Emanuel Zgraggen, Arvind Satyanarayan, Tim Kraska, Çagatay Demiralp, and César Hidalgo. 2019. Sherlock: A Deep Learning Approach to Semantic Data Type Detection. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM.
  30. [30]Nguyen Hung, Tam Nguyen, Zoltan Miklos, Karl Aberer, Avigdor Gal, and Matthias Weidlich. 2014. Pay-as-you-go Reconciliation in Schema Matching Networks. Proceedings - International Conference on Data Engineering, 220–231. https://doi.org/10.1109/ICDE.2014.6816653
  31. [31]Hiroshi Iida, Dung Thai, Varun Manjunatha, and Mohit Iyyer. 2021. TABBIE: Pretrained Representations of Tabular Data. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tür, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmay Chakraborty, and Yichao Zhou (Eds.). Association for Computational Linguistics, Online, 3446–3456. https://doi.org/10.18653/v1/2021.naacl-main.270
  32. [32]Aamod Khatriwada, Grace Fan, Roee Shraga, Zixuan Chen, Wolfgang Gatterbauer, Renée J. Miller, and Mirek Riedewald. 2023. SANTOS: Relationship-based Semantic Table Union Search. Proc. ACM Manag. Data 1, 1, Article 9 (May 2023), 25 pages. https://doi.org/10.1145/3588689
  33. [33]Christos Koutras, George Siachamis, Andra Ionescu, Kyriakos Parasakis, Jerry Brons, Marios Fragkoulis, Christoph Lofi, Angela Bonifati, and Asterios Katsifodimos. 2020. Valentine: Evaluating Matching Techniques for Dataset Discovery. 2021 IEEE 37th International Conference on Data Engineering (ICDE) (2020), 468–479. https://api.semanticscholar.org/CorpusID:222378204arxiv.org/abs/2010.07386
  34. [34]Sven Langeneker, Christoph Sturm, Christian Schalles, and Carsten Binnig. 2023. SportsTables: A New Corpus for Semantic Type Detection (Extended Version). Datenbank-Spektrum 23 (10 2023). https://doi.org/10.1007/s13222-023-00457-y
  35. [35]Sven Langeneker, Christoph Sturm, Christian Schalles, and Carsten Binnig. 2024. Pythagoras: Semantic Type Detection of Numerical Data in Enterprise Data Lakes. In Proceedings of the 27th International Conference on Extending Database Technology (EDBT) (Paestum, Italy). OpenProceedings.org, 725–733.
  36. [36]Maurizio Lenzerini. 2002. Data integration: a theoretical perspective. In Proceedings of the Twenty-First ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems (Madison, Wisconsin) (PODS ’02). Association for Computing Machinery, New York, NY, USA, 233–246. https://doi.org/10.1145/543613.543644
  37. [37]Yuanxi Li and Allan Tucker. 2014. Integrating Clinical Data from Cross-Sectional and Longitudinal Studies. In 2014 IEEE 27th International Symposium on Computer-Based Medical Systems. 465–466. https://doi.org/10.1109/CBMS.2014.92
  38. [38]Yurong Liu, Eduardo H. M. Pena, Aécio Santos, Eden Wu, and Juliana Freire. 2025. Magneto: Combining Small and Large Language Models for Schema Matching. Proc. VLDB Endow. 18, 8 (April 2025), 2681–2694. https://doi.org/10.14778/3742728.3742757
  39. [39]Jose Lugo-Martinez, Siwei Xu, Justine Levesque, Daniel Gallagher, Leslie A. Parker, Josef Neu, Christopher J. Stewart, Janet E. Berrington, Nicholas D. Embleton, Gregory Young, Katherine E. Gregory, Misty Good, Arti Tandon, David Genetti, Tracy Warren, and Ziv Bar-Joseph. 2022. Integrating longitudinal clinical and microbiome data to predict growth faltering in preterm infants. Journal of Biomedical Informatics 128 (2022), 104031. https://doi.org/10.1016/j.jbi.2022.104031
  40. [40]Jayant Madhavan, Philip A. Bernstein, AnHai Doan, and Alon Halevy. 2005. Corpus-Based Schema Matching. In Proceedings of the 21st International Conference on Data Engineering (ICDE ’05). IEEE Computer Society, USA, 57–68. https://doi.org/10.1109/ICDE.2005.39
  41. [41]Jayant Madhavan, Philip A. Bernstein, and Erhard Rahm. 2001. Generic Schema Matching with Cupid. In Proceedings of the 27th International Conference on Very Large Data Bases (VLDB ’01). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 49–58.
  42. [42]Vishwanadham Mandala. 2017. Data Integration and Data Engineering Techniques. International Journal of Scientific Research and Management (IJSRM) 5, 5 (2017), 5354–5359. https://doi.org/10.18535/ijsrm/v5i5.13
  43. [43]Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schütze. 2008. Introduction to Information Retrieval. Cambridge University Press, USA.
  44. [44]Katerina M. Marcoulides and Kevin J. Grimm. 2017. Data Integration Approaches to Longitudinal Growth Modeling. Educational and Psychological Measurement 77, 6 (2017), 971–989. https://doi.org/10.1177/0013164416664117 arXiv:https://doi.org/10.1177/0013164416664117
  45. [45]Marianti Mukhopadhyay, Oliver Kennedy, Michael Brachmann, Raktim Mukhopadhyay, Arpan Dharia, and Andrew H. Talal. 2023. Social determinants of health derived from people with opioid use disorder: Improving data collection, integration and use with cross-domain collaboration and reproducible, data-centric, notebook-style workflows. Frontiers in Medicine Volume 10 - 2023 (2023). https://doi.org/10.3389/fmed.2023.1067964
  46. [46]Margherita Martorana, Tobias Kuhn, Ronald Siebes, and Jacco VanOssenbruggen. 2022. Aligning restricted access data with FAIR: a systematic review. PeerJ Computer Science (Jul 20 2022). https://www.proquest.com/scholarly-journals/aligning-restricted-access-data-with-fair/docview/2691902499/se-2 Copyright - © 2022 Martorana et al. This is an open access article distributed under the terms of the Creative Commons Attribution License: https://creativecommons.org/licenses/by/4.0/ (the “License”), which permits unrestricted use, distribution, reproduction and adaptation in any medium and for any purpose provided that it is properly attributed. For attribution, the original author(s), title, publication source (PeerJ Computer Science) and either DOI or URL of the article must be cited. Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License; Last updated - 2023-12-01.DOI
  47. [47]Margherita Martorana, Tobias Kuhn, and Jacco van Ossenbruggen. 2025. Metadata-driven Table Union Search: Leveraging Semantics for Restricted Access Data Integration. arXiv:2502.20945 [cs.DB] https://arxiv.org/abs/2502.20945
  48. [48]Sergey Melnik, Hector Garcia-Molina, and Erhard Rahm. 2002. Similarity Flooding: A Versatile Graph Matching Algorithm and Its Application to Schema Matching. In Proceedings of the 18th International Conference on Data Engineering (ICDE ’02). IEEE Computer Society, USA, 117.DOI
  49. [49]Saptashwa Mitra, Yu Qiu, Haley Moss, Kaigang Li, and Sangmi Lee Pallickara. 2018. Effective Integration of Geotagged, Ancillary Longitudinal Survey Datasets to Improve Adulthood Obesity Predictive Models. In 2018 17th IEEE International Conference On Trust, Security And Privacy In Computing And Communications/ 12th IEEE International Conference On Big Data Science And Engineering (Trust-Com/BigDataSE). 1738–1746. https://doi.org/10.1109/TrustCom/BigDataSE.2018.00263
  50. [50]Bylah Mugotitsa, Thathagata Bhattacharjee, Michael Ochola, Dorothy Maiosli, David Amadi, Pauline Andeso, Joseph Kuria, Reinpreet Momanyi, Evans Omondi, Dan Kajungu, Jim Todd, Agnes Kiragga, and Jay Greenfield. 2024. Integrating longitudinal mental health data into a staging database: harnessing DDI-lifecycle and OMOP vocabularies within the INSPIRE Network Datahub. Frontiers in Big Data Volume 7 - 2024 (2024). https://doi.org/10.3389/fdata.2024.1435510
  51. [51]Fatemeh Nargesían, Erkang Zhu, Ken Q. Pu, and Ren ée J. Miller. 2018. Table union search on open data. Proc. VLDB Endow. 11, 7 (March 2018), 813–825. https://doi.org/10.14778/3192965.3192973
  52. [52]van Tuan Nguyen. 2024. Integrating longitudinal data for enhanced survival analysis : methods and applications. Theses. Universite Paris Cité. https://theses.hal.science/tel-04903680DOI
  53. [53]Koyena Pal, Aamod Khatiwada, Roee Shraga, and Renée J Miller. [n. d.]. ALT-GEN: Benchmarking Table Union Search using Large Language Models. Proceedings of the VLDB Endowment. ISSN 2150 ([n. d.]), 8097.
  54. [54]Jeff Z. Pan, Elspeth Edelstein, Patrik Bansky, and Adam Wyner. 2021. A Knowledge Graph Based Approach to Social Science Surveys. Data Intelligence 3, 4 (10 2021), 477–506. https://doi.org/10.1162/dint_a_00107
  55. [55]Marcel Parzick, Brecht Vandevoort, Frank Neven, Liesbet M. Peeters, and Stijn Vansummeren. 2024. Schema Matching with Large Language Models: an Experimental Study. arXiv:2407.11852 [cs.DB] https://arxiv.org/abs/2407.11852
  56. [56]Pratik Pokharel, Juseung Lee, Oliver Kennedy, Marianti Markatou, Andrew Talal, Jeff Good, and Raktim Mukhopadhyay. 2024. Drag, Drop, Merge: A Tool for Streamlining Integration of Longitudinal Survey Instruments. In Proceedings of the 2024 Workshop on Human-In-the-Loop Data Analytics (HILDA ’24). Association for Computing Machinery, New York, NY, USA, 1–7. https://doi.org/10.1145/3666599.3665965
  57. [57]Rachel Pottinger and Philip A. Bernstein. 2003. Merging Models Based on Given Correspondences. In Proceedings of the 29th International Conference on Very Large Data Bases (VLDB). VLDB Endowment, 826–837. https://www.vldb.org/conf/2003/papers/S26P01.pdfDOI
  58. [58]Christoph Quix, David Kensche, and Xiang Li. 2007. Generic Schema Merging. In Proceedings of the 19th International Conference on Advanced Information Systems Engineering (CAiSE) (Lecture Notes in Computer Science, Vol. 4495). Springer, 127–141. https://doi.org/10.1007/978-3-540-72988-4_10
  59. [59]Erhard Rahm and Philip A. Bernstein. 2001. A survey of approaches to automatic schema matching. The VLDB Journal 10, 4 (Dec. 2001), 334–350. https://doi.org/10.1007/s007780100057
  60. [60]Thomas Rebele, Fabian Suchanek, Johannes Hoffart, Joanna Biega, Erdal Kuzey, and Gerhard Weikum. 2016. YAGO: A Multilingual Knowledge Base from Wikipedia, Wordnet, and Geonames. In The Semantic Web – ISWC 2016: 15th International Semantic Web Conference, Kobe, Japan, October 17–21, 2016, Proceedings, Part II (Kobe, Japan). Springer-Verlag, Berlin, Heidelberg, 177–185. https://doi.org/10.1007/978-3-319-46547-0_19
  61. [61]Diego Rodrigues and Alitgran Silva. 2021. A study on machine learning techniques for the schema matching network problem. Journal of the Brazilian Computer Society 27 (12 2021). https://doi.org/10.1186/s13173-021-00119-5
  62. [62]Mario Scrocca, Damiano Scandolari, Gloria Re Calegari, Ilaria Baroni, and Irene Celino. 2021. The Survey Ontology: Packaging Survey Research as Research Objects. In Proceedings of the 2nd Workshop on Data and Research Objects Management for Linked Open Science - co-located with ISWC 2021. https://doi.org/10.4126/FRL01-006429412
  63. [63]Kavitha Srinivas, Julian Dolby, Ibrahim Abdelaziz, Oktie Hassanzadeh, Harsha Kokei, Aamod Khatiwada, Tejaswini Pedapati, Subhajit Chaudhury, and Horst Samulowitz. 2023. LakeBench: Benchmarks for Data Discovery over Data Lakes. arXiv:2307.04217 [cs.DB] https://arxiv.org/abs/2307.04217
  64. [64]Weifeng Su, Jiying Wang, and Frederick Lochovsky. 2006. Holistic Schema Matching for Web Query Interfaces. 77–94. https://doi.org/10.1007/11687238_8
  65. [65]Yoshihiko Suhara, Jinfeng Li, Yuliang Li, Dan Zhang, Çağatay Demiralp, Chen Chen, and Wang-Chieh Yew. 2022. Annotating Columns with Pre-trained Language Models. In Proceedings of the 2022 International Conference on Management of Data (Philadelphia, PA, USA) (SIGMOD ’22). Association for Computing Machinery, New York, NY, USA, 1493–1503. https://doi.org/10.1145/3514221.3517906
  66. [66]Yongkang Sun, Zhihao Ding, Huiqiang Wang, Reynold Cheng, and Jieming Shi. 2026. Efficient and Effective Table-Centric Table Union Search in Data Lakes. arXiv:2603.17298 [cs.DB] https://arxiv.org/abs/2603.17298
  67. [67]Avijad Susman, Rupak Krishnamurthy, Mohammed Al Olainat, Serdar Bozdag, Bino Varghese, Nasim Sheikh-Bahaei, and Gaurav Pandey. 2025. Longitudinal Ensemble Integration for Sequential Classification with Multimodal Data. In 2025 IEEE International Conference on Digital Health (ICDH). 31–40. https://doi.org/10.1109/ICDH67620.2025.00014
  68. [68]Mimi Haryani Tasani and Mohd Kamar Yusuf. 2025. Challenges in Data Integration for Different Data Types: A Systematic Review. In 2025 4th International Conference on Computer Technologies (ICCTech). 98–102. https://doi.org/10.1109/ICCTech66294.2025.00027
  69. [69]Lauren A. Taylor, Annabel Xulin Tan, Caitlin E. Coyle, Chima Ndumele, Erika Rogan, Maureen Canavan, Leslie A. Curry, and Elizabeth H. Bradley. 2016. Leveraging the Social Determinants of Health: What Works? PLOS ONE 11, 8 (Aug. 2016), e0160217. https://doi.org/10.1371/journal.pone.0160217
  70. [70]Nicholas Thieberger and Andrea L. Berez. 2012. Linguistic data management. In The Oxford handbook of linguistic fieldwork, Nicholas Thieberger (Ed.). Oxford: OUP, 90–118.DOI
  71. [71]Caitlin Thomas-Henkel and Meryl Schulman. 2017. Screening for Social Determinants of Health in Populations with Complex Needs: Implementation Considerations.
  72. [72]Jianhong Tu, Ju Fan, Nan Tang, Peng Wang, Guoliang Li, Xiaoyong Du, Xiaofeng Jia, and Song Gao. 2023. Unicorn: A Unified Multi-tasking Model for Supporting Matching Tasks in Data Integration. Proc. ACM Manag. Data 1, 1, Article 84 (May 2023), 26 pages. https://doi.org/10.1145/3588938
  73. [73]Mohamed Yakoubi, Kris Ganjam, Kausar Chakrabarti, and Surajit Chaudhuri. 2012. InfoGather: entity augmentation and attribute discovery by holistic matching with web tables. In Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data (Scottsdale, Arizona, USA) (SIGMOD ’12). Association for Computing Machinery, New York, NY, USA, 97–108. https://doi.org/10.1145/2213836.2213848
  74. [74]Dan Zhang, Yoshihiko Suhara, Jimfeng Li, Madelon Hulsebos, Çağatay Demiralp, and Wang-Chieh Tan. 2020. Contextual Semantic Type Detection in Tables. Proc. VLDB Endow. 13, 12 (2020), 1835–1848. https://doi.org/10.14778/3407790.3407793
  75. [75]Meihui Zhang, Marios Hadjieleftheriou, Beng Chin Ooi, Cecilia M. Procopiu, and Divesh Srivastava. 2011. Automatic discovery of attributes in relational databases. In Proceedings of the 2011 ACM SIGMOD International Conference on Management of Data (Athens, Greece) (SIGMOD ’11). Association for Computing Machinery, New York, NY, USA, 109–120. https://doi.org/10.1145/1989323.1989336
  76. [76]Yi Zhang, Peter Baile Chen, and Zachary G. Ives. 2024. Searching Data Lakes for Nested and Joined Data. Proc. VLDB Endow. 17, 11 (July 2024), 3346–3359. https://doi.org/10.14778/3681954.3682005
  77. [77]Erkang Zhu, Dong Deng, Fatemeh Nargesiani, and Renée J. Miller. 2019. JOSIE: Overlap Set Similarity Search for Finding Joinable Tables in Data Lakes. In Proceedings of the 2019 International Conference on Management of Data (Amsterdam, Netherlands) (SIGMOD ’10). Association for Computing Machinery, New York, NY, USA, 847–864. https://doi.org/10.1145/3299869.3300065

Paper details

vlnc:W_khpre5cr
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents