WorksComputer Science
Federated Dataset Simulation: Inducing Label-Free Heterogeneity Across Tasks
Abstract
Federated Learning enables collaborative model training across decentralized clients without sharing raw data. While the field has seen rapid growth, the datasets underpinning FL evaluation have not kept pace: real federated datasets remain scarce, and most studies rely on small-scale centralized classification datasets partitioned via Dirichlet label skew, a one-dimensional proxy for the multi-dimensional heterogeneity present in real federations. Moreover, label-based partitioning is by construction inapplicable to non-classification tasks such as segmentation and detection, for which no principled simulation method exists. We cast the conversion of centralized datasets into federated benchmarks as the task of Federated Dataset Simulation (FDS), and decompose it into a representation function and an assignment mechanism . We provide two label-free instantiations – geometric partitioning via pretrained model embeddings and semantic partitioning via vision-language model-derived criteria – applicable to classification and dense prediction tasks alike. We evaluate across four datasets spanning classification, segmentation, and detection, and show that both methods induce heterogeneity that matches or exceeds that of label-skew partitioning, with measurable impact on downstream FL performance. We release fds, a modular open-source package for end-to-end federated dataset simulation, heterogeneity measurement, and FL training, at https://github.com/siomvas/fds.
Introduction
Federated learning (FL) [36] is a collaborative training paradigm that has matured into a central solution for privacy-preserving and decentralized ML, with production deployments in several industries [27]. Despite an exponential growth in research output, a well-documented gap persists in the scale, complexity, and diversity of the datasets and tasks underpinning FL evaluation [37, 23]: the vast majority of empirical studies are conducted on image classification at the scale and complexity of CIFAR [26]. While naturally federated datasets exist [7, 38, 56], expanding this dataset pool is laborious and costly, and the heterogeneity such datasets exhibit is often idiosyncratic and not transferable to other tasks or domains. It has therefore become standard practice to partition existing centralized datasets into virtual clients for FL evaluation [46]. For a virtual partition to provide a challenging FL evaluation, the consensus is it must reflect a central tenet of FL theory, namely that federated optimization performance degrades as the statistical heterogeneity (non-IIDness) of client data increases [36, 50].
This creates a need for methods simulating non-IIDness across the virtual clients of a split centralized dataset. The prevailing approach to do so is Dirichlet-based label partitioning [46], which assigns skewed class proportions across clients by sampling from a Dirichlet distribution [22]. This reduces the multi-dimensional heterogeneity (covariate shift, concept shift, domain shift) observed in real federations to a single axis, when in practice clients may differ not only in which classes they observe but in how those classes appear [24]. Moreover, Dirichlet partitioning operates on sample-level class labels, so is by construction inapplicable to any task where samples are not uniquely associated with a single category (e.g., segmentation, machine translation, and sound event detection). For these tasks, which constitute a growing share of practical FL applications, no principled heterogeneity simulation method exists, and practitioners are limited to random partitioning or ad-hoc splits [23]. As a result, current FL evaluations and benchmarks produce conclusions about FL research progress (in performance, convergence rates, fairness, and robustness) that may not generalize to real deployments.
We cast this problem as the task of Federated Dataset Simulation (FDS): converting a centralized dataset into a federated benchmark in a controllable manner, making heterogeneity simulation itself an object of systematic design and measurement. To enable simulation that is agnostic to the downstream task and label format, we decompose FDS into two sub-tasks: a representation function , which maps data into a structured latent space that captures axes of variation related to heterogeneity, and an assignment mechanism , which allocates samples to virtual clients within that space. This decomposition provides a unifying formalism under which existing approaches, like label Dirichlet partitioning, can be expressed as special cases. Because operates on raw inputs rather than annotations, the framework is naturally label-free and modality-agnostic. We provide and evaluate two concrete instantiations of this framework in the vision domain, each targeting a different latent space. The first, geometric partitioning, defines as the embedding function of a self-supervised pretrained foundation model and uses -means clustering as the assignment . Because such encoders are trained to organize semantically related inputs into coherent regions of their representation space, partitioning along geometric boundaries in this space induces covariate shift that reflects meaningful visual and semantic structure. The second, semantic partitioning, defines as a chain of pretrained vision-language and large language models which extracts per-sample sets of human-interpretable semantic criterion labels. A Dirichlet-weighted assignment then correlates client identity with specific criterion values to produce federations where each client can be characterized by the visual properties they tend to exhibit.
We validate the framework across four datasets encompassing classification, semantic segmentation, and object detection, and benchmark FL performance under a varying number of clients. We demonstrate that both geometric and semantic partitioning produce distinct heterogeneity profiles that surpass existing approaches, not only in terms of feature and label (where applicable) heterogeneity, but also on FL convergence. Our contributions are as follows:
Task formulation. We formalize federated dataset simulation through the decomposition, establishing a unified framework in which heterogeneity type, severity, and measurement are jointly specified and become the object of study and design.
Partitioning methods. We introduce two distinct label-free partitioning pipelines that induce feature-level heterogeneity and apply to classification and dense prediction tasks alike, resulting in similar or greater heterogeneity than Dirichlet partitioning despite using no labels.
Empirical evaluation and benchmark. We produce a multi-faceted evaluation spanning label and feature space heterogeneity along with FL performance and validate the framework across four challenging vision datasets, showing that the proposed partitioning methods result in a more challenging setting that can serve as a better benchmark for FL algorithm development.
Software package. With the intention of lowering the barrier for researchers to move beyond label-skew-only evaluation and classification, we release
fds, a modular open-source library providing an end-to-end pipeline for federated dataset simulation: feature extraction across multiple pretrained encoders, semantic clustering based on our VLM-LLM pipeline, controllable partitioning to clients, heterogeneity measurement in the feature or label space, and integrated downstream FL training for classification, segmentation, or detection.
The FDS Framework
General Problem Formulation
Definition 1 (Federated Dataset Simulation). Let denote a centralized dataset comprising samples, where and represent the input and annotation space, respectively. The Federated Dataset Simulation (FDS) task seeks a partition such that: (all samples are assigned), (samples are uniquely assigned), and the induced local distributions exhibit meaningful statistical heterogeneity.
Definition 2 (Partitioning Mechanism). Given , a partitioning mechanism consists of:
A representation function , which projects inputs into a structured latent space .
An assignment mechanism which produces a partitioning from the set of latent representations .
The assignment mechanism operates via a set of client parameters , which may be computed from the data, sampled from a prior, or both, and then allocates samples to clients based on the affinity between each sample’s representation and the client parameters. Label-skew Dirichlet partitioning [22] fits this framework as the special case where , and samples of each class are allocated to clients in proportion to . The concentration parameter controls heterogeneity from IID () to near class-disjoint ().
Unlike the Dirichlet special case, the two instantiations we propose next operate on alone, making them applicable to settings where sample-level labels are unavailable, noisy, or dense (e.g., segmentation and detection tasks). This also extends the space of heterogeneity that can be expressed: because the label space is a lossy projection of , feature-based representations can capture distributional structure that label-based partitioning cannot.
Definition 3 (Heterogeneity Functional). Given a realized partition, we quantify heterogeneity via the mean pairwise divergence of client distributions in the representation space:
where denotes the empirical distribution of for samples assigned to client , and is a divergence measure appropriate to .
Label Distribution Heterogeneity When sample-level class labels are available, we examine how client label distributions diverge. We measure this via the Jensen–Shannon divergence, a symmetric, bounded measure well-suited to discrete distributions:
Feature-distribution heterogeneity To measure heterogeneity in a continuous representation space, we use the Maximum Mean Discrepancy (MMD) [17] with an RBF kernel and the Frechet Inception Distance (FID) [21], computed in the activation space of a frozen InceptionV3 network [47] independent of the embedding encoder used for partitioning, avoiding the circularity of evaluating partitions in the same embedding space used to create them:
Geometric Partitioning
This instantiation operates in the continuous feature space induced by pretrained foundation models. Let be a model pretrained with a self-supervised or vision-language objective. We define as . The assignment mechanism partitions the embedding space via -means clustering with clusters equal to the number of target clients. The client parameters are the cluster centroids , computed from the full set of representations, and each sample is assigned to its nearest centroid .
Modern pretrained representations, whether trained via self-distillation [39] or vision-language alignment [43], organize semantically related inputs into coherent regions of the embedding space, so partitioning along geometric boundaries in this space induces covariate shift that reflects the visual and semantic structure captured by the encoder. The resulting clients specialize in different visual domains, object types, or scene compositions, depending on which axes of variation are salient in the encoder’s representation space, without requiring access to annotations. The number of clients acts as an implicit heterogeneity knob, where fewer clients produce broader clusters spanning more diverse manifold regions, while more clients yield finer partitions with greater inter-client divergence. We evaluate multiple encoder families and token aggregation strategies in Section 3 and report the selection procedure used to identify a recommended default configuration.
VLM-based Semantic Partitioning
The second instantiation operates over a discrete criterion space , where each is a finite set of categorical values corresponding to a human-interpretable semantic criterion (e.g., cuisine origin or texture for a food photography dataset). The representation function maps each image to a tuple of criterion labels , and the assignment mechanism allocates samples to clients based on these discrete representations using a product-of-Dirichlets mechanism.
Criterion label extraction is a multi-stage pipeline chaining a vision-language model (VLM) and a large language model (LLM) to perform text-conditioned image clustering. For each dataset, semantically meaningful axes of variation are identified through domain inspection, and candidate criteria are validated by querying the VLM on a sample subset to verify non-degenerate label distributions. After is established, per-sample labeling proceeds as an iterative procedure, inspired by Kwon et al. [28]: the VLM is prompted to produce a short description of each image focusing exclusively on criterion . An LLM converts each description into a single-word raw label summarizing the image with respect to to form a dictionary. The LLM then clusters this dictionary of raw labels into canonical categories to consolidate semantically close concepts. Each sample description is then classified into one of these categories, yielding the criterion label , using the same LLM. This is repeated for each criterion to construct .
Assignment mechanism. Given the extracted criterion vectors, samples are assigned to clients via a product-of-Dirichlets scheme. For each client and each criterion , a preference vector is drawn. The assignment probability of sample to client is , where denotes the preference weight that client places on value of criterion . This multiplicative composition ensures that a sample receives high affinity for a client only if it matches that client’s preferences across all criteria simultaneously. This mechanism generalizes the standard Dirichlet label-skew assignment to the multi-label setting. The concentration serves as the heterogeneity knob, like in the typical label-skew case. The full set of criteria and prompt details used per dataset are reported in Appendix C and included in the code release. As this is a computationally expensive pipeline, our code release also includes the precomputed criterion labels for all 4 evaluated datasets.
Connection to Federated Learning Performance
The previous sections define methods for inducing distributional divergence across virtual clients, but not all such divergence is equally consequential for federated optimization. A partition that separates clients along axes to which the model can easily learn invariant features may produce large while having negligible impact on model convergence. The optimization difficulty induced by a partition depends not only on the magnitude of but on the mutual information between the representation and the task: when heterogeneity in does not translate into optimization difficulty as opposed to when is high. This clarifies why ad-hoc heterogeneity methods such as additive Gaussian noise [30] or geometric rotations [14] do not typically produce meaningful FL difficulty especially if the training pipeline includes augmentations for such perturbations. To effectively benchmark FL algorithms across different tasks, client configurations, and optimization strategies, we therefore need partitioning methods that produce heterogeneity with high .
Table 1: Heterogeneity metrics on classification datasets across client counts . JSD measures label-distribution divergence; MMD, FID, and centroid distance measure feature-distribution divergence (computed in InceptionV3 activation space). Higher values indicate greater heterogeneity. Geometric partitioning uses no labels yet induces the highest heterogeneity across all metrics, including JSD.
| Dataset | Method | JSD 5 | 10 | 20 | MMD () 5 | 10 | 20 | FID 5 | 10 | 20 | Centroid 5 | 10 | 20 |
| Tiny | Random | .003 | .005 | .010 | 0.00 | 0.00 | 0.00 | 3.3 | 6.6 | 13.2 | 0.9 | 0.9 | 0.9 |
| Tiny | Label Dir. | .559 | .572 | .569 | 0.40 | 0.55 | 0.75 | 29.2 | 39.0 | 49.4 | 4.9 | 6.0 | 7.0 |
| Tiny | Semantic | .128 | .168 | .207 | 0.70 | 0.93 | 1.19 | 22.0 | 30.9 | 43.0 | 4.8 | 5.5 | 6.2 |
| Tiny | Semantic | .152 | .240 | .315 | 0.58 | 1.33 | 2.15 | 23.2 | 42.1 | 62.2 | 4.8 | 6.7 | 8.7 |
| Tiny | Geometric | .630 | .655 | .668 | 5.31 | 9.70 | 18.0 | 111 | 146 | 201 | 13.8 | 21.6 | 31.1 |
| Food | Random | .002 | .003 | .007 | 0.00 | 0.00 | 0.00 | 2.7 | 5.4 | 10.5 | 0.8 | 0.9 | 1.0 |
| Food | Label Dir. | .564 | .587 | .566 | 0.69 | 1.06 | 1.06 | 16.5 | 24.6 | 29.9 | 8.3 | 9.9 | 11.1 |
| Food | Semantic | .138 | .168 | .215 | 0.67 | 0.75 | 1.10 | 12.5 | 16.3 | 25.7 | 8.0 | 8.2 | 9.7 |
| Food | Semantic | .213 | .284 | .359 | 0.61 | 1.50 | 2.47 | 14.3 | 25.3 | 39.8 | 8.3 | 11.2 | 14.3 |
| Food | Geometric | .620 | .660 | .664 | 6.58 | 16.4 | 13.8 | 74.0 | 127 | 122 | 27.0 | 39.0 | 40.5 |
Table 1.
Table 2: Heterogeneity metrics on dense prediction datasets. Dirichlet label-skew partitioning is inapplicable to segmentation and detection, so Random is the only existing baseline. Geometric and semantic partitioning are the first methods to produce non-trivial heterogeneity for these tasks.
| Dataset | Method | MMD () 5 | 10 | 20 | FID 5 | 10 | 20 | Centroid 5 | 10 | 20 |
| ADE20K | Random | 0.00 | 0.00 | 0.01 | 11.1 | 21.8 | 41.3 | 0.9 | 1.3 | 1.8 |
| ADE20K | Semantic | 2.13 | 2.27 | 2.16 | 47.1 | 60.8 | 80.5 | 6.4 | 6.8 | 7.0 |
| ADE20K | Semantic | 2.80 | 4.56 | 5.10 | 59.4 | 92.6 | 124.6 | 8.1 | 10.1 | 11.0 |
| ADE20K | Geometric | 13.1 | 16.5 | 19.0 | 167 | 191 | 247 | 20.7 | 25.6 | 31.9 |
| Pascal VOC | Random | 0.01 | 0.02 | 0.02 | 52.3 | 87.4 | 129.6 | 2.0 | 2.8 | 3.9 |
| Pascal VOC | Semantic | 0.68 | 1.21 | 1.29 | 68.6 | 112.2 | 152.1 | 5.1 | 6.7 | 7.6 |
| Pascal VOC | Semantic | 1.09 | 1.52 | 2.16 | 79.8 | 120.5 | 170.6 | 6.3 | 7.5 | 9.4 |
| Pascal VOC | Geometric | 18.1 | 16.7 | 30.1 | 219 | 228 | 267 | 22.0 | 23.6 | 29.7 |
Table 2.
Both proposed instantiations are designed to meet this requirement. is extracted from encoders whose pretraining objectives aim to preserve semantic content, producing representations with high across a wide range of vision tasks [4]. Partitioning in this space induces distributional shifts that the model is likely to be sensitive to, which we verify empirically in Section 3. For , task-coupling depends on the choice of criteria, with criteria that describe properties correlated with the target variable (e.g., main object type, scene composition) producing higher than criteria more orthogonal to the task, as we show via criterion-level ablation in Section 3. This reflects an asymmetry between the two instantiations: geometric partitioning derives task-relevance implicitly from the pretraining objective, while semantic partitioning requires the explicit design of task-relevant criteria but offers interpretability, as the axes of heterogeneity are human-readable.
Evaluation
Evaluation Design
Encoder and model selection. Our geometric partitioning method depends on the choice of representation function . We therefore evaluate several pretrained vision encoders, including DINOv2 [39], CLIP [43], and MAE [20] variants across several token extraction and aggregation strategies, and select a single default configuration based on average rank of MMD across datasets. Unless otherwise stated, we use DINOv2-B features obtained by concatenating the CLS tokens from the final four layers, followed by UMAP [35] dimensionality reduction before clustering. This choice provided a strong and computationally tractable default across classification, segmentation, and detection tasks. For semantic partitioning, we use the open-source Intern-VL3.5 [51] and Llama 3.3 70B [16] models as the VLM and LLM components, respectively, in order to keep the pipeline reproducible and accessible under a modest compute budget; all extracted criterion labels are cached and reused across experiments.
Datasets and tasks. We evaluate on four vision datasets spanning three prediction settings. Tiny-ImageNet [29] provides a more visually complex classification benchmark than CIFAR-scale datasets while remaining comparable to prior FL work. Food101 [6] introduces a fine-grained, domain-specific classification task with substantial intra-class variation and label noise. To test whether FDS extends beyond sample-level class labels, we further include ADE20K [58] for semantic segmentation and PASCAL VOC [12] for object detection. These dense prediction tasks are important for our setting because standard label Dirichlet partitioning is not naturally defined when each sample contains multiple labels, spatially structured labels, or object sets rather than a single class.
FL training protocol. For each dataset, we compare random partitioning, label Dirichlet partitioning where applicable, geometric partitioning, and semantic partitioning. We vary the number of clients to study how each partitioning mechanism scales with federation size. All FL experiments use FedAvg [36] with 1 local epoch for 100 communication rounds. We use a ResNet-18 [19], DeepLabv3 with ResNet-50 [8] for semantic segmentation, and a Faster R-CNN model with a ResNet-50-FPN backbone [32] for detection. All models are initialized from pretrained weights. We tune dataset-specific optimization hyperparameters on a held-out subset using Optuna [2] and then fix them across partitioning methods.
Metrics. We evaluate each partition along three axes. We measure feature-space heterogeneity using the MMD, FID, and feature centroid Euclidean distance averaged over pairwise distances between client distributions in the encoder representation space. Where sample-level labels are available, we report label-space heterogeneity using Jensen–Shannon divergence between client label distributions. Downstream FL performance is evaluated using task-appropriate metrics: top-1 accuracy for Tiny ImageNet and Food101, mIoU for ADE20K, and mAP@50 for PASCAL VOC. This separates the question of whether a partition induces measurable distributional heterogeneity from whether that heterogeneity produces optimization difficulty for FL.
Main Results
Induced Heterogeneity Tables 1 and 2 show that the proposed FDS instantiations induce feature-space heterogeneity across tasks. On classification datasets, label Dirichlet partitioning strongly increases label-distribution divergence but has comparatively modest effect in feature space. In contrast, geometric partitioning, despite using no labels, produces the largest feature-space divergence and also induces high label divergence as a byproduct of partitioning semantically structured representations. Semantic partitioning occupies an intermediate regime: it produces non-random feature heterogeneity with an interpretable severity parameter, and the setting consistently increases heterogeneity relative to . On dense prediction datasets both geometric and semantic partitioning produce non-trivial feature heterogeneity relative to random splits, supporting the use of the framework beyond classification.
Impact on downstream FL performance. The heterogeneity induced by FDS translates into measurable downstream optimization difficulty. Table 3 reports global validation performance under FedAvg for clients. Across all datasets and client counts, non-random partitions degrade FedAvg performance relative to random splitting, indicating that the induced client differences are not merely artifacts of the heterogeneity metrics. Geometric partitioning is the most challenging setting in most configurations, producing the largest degradation on all Tiny ImageNet, ADE20K, and PASCAL VOC settings, and on Food101 for and . Semantic partitioning is generally less severe but remains consistently harder than random, while preserving an interpretable -controlled assignment mechanism. On classification datasets, label-Dirichlet skew remains a strong baseline at ; however, on dense prediction tasks it is not directly applicable, whereas both proposed FDS instantiations continue to induce non-trivial performance degradation.
| Dataset | Metric | Partition | |||
| Tiny ImageNet | Acc. | Random | 74.53 (0.00) | 74.63 (0.00) | 74.29 (0.00) |
| Label Dir. | 71.74 (2.79) | 71.56 (3.07) | 71.74 (2.55) | ||
| Label Dir. | 74.30 (0.23) | 74.19 (0.44) | 73.57 (0.72) | ||
| Semantic | 73.69 (0.84) | 70.43 (4.20) | 72.99 (1.30) | ||
| Semantic | 73.56 (0.97) | 71.32 (3.31) | 72.31 (1.98) | ||
| Geometric | 67.41 (7.12) | 64.84 (9.79) | 69.24 (5.05) | ||
| Food101 | Acc. | Random | 77.16 (0.00) | 74.43 (0.00) | 70.39 (0.00) |
| Label Dir. | 71.31 (5.85) | 66.45 (7.98) | 64.20 (6.19) | ||
| Label Dir. | 75.75 (1.41) | 72.89 (1.54) | 68.80 (1.59) | ||
| Semantic | 72.93 (4.23) | 64.49 (9.94) | 63.97 (6.42) | ||
| Semantic | 74.93 (2.23) | 71.66 (2.77) | 65.68 (4.71) | ||
| Geometric | 70.02 (7.14) | 63.15 (11.28) | 66.34 (4.05) | ||
| ADE20K | mIoU | Random | 40.23 (0.00) | 39.02 (0.00) | 36.91 (0.00) |
| Semantic | 38.13 (2.10) | 36.74 (2.28) | 33.46 (3.45) | ||
| Semantic | 38.98 (1.25) | 37.82 (1.20) | 34.01 (2.90) | ||
| Geometric | 35.37 (4.86) | 31.14 (7.88) | 29.34 (7.57) | ||
| PASCAL VOC | mAP@50 | Random | 79.40 (0.00) | 83.10 (0.00) | 82.14 (0.00) |
| Semantic | 78.15 (1.25) | 77.56 (5.54) | 77.99 (4.15) | ||
| Semantic | 78.12 (1.28) | 78.12 (4.98) | 79.22 (2.92) | ||
| Geometric | 74.34 (5.06) | 73.19 (9.91) | 78.10 (4.04) |
Table 3. Downstream FL performance under FedAvg across datasets and client counts. Parentheses report degradation relative to random partitioning. Lower non-random performance within each dataset/client-count block in bold. Label Dirichlet baselines are omitted for dense prediction tasks because they do not provide a single sample-level class label.
Ranking of Aggregation Methods
We compare seven aggregation methods, FedAvg [36], FedDyn [11], FedNova [49], FedProx [31], FedSAM [42], SCAFFOLD [25], and FedLC [57], under label Dirichlet skew and semantic partitioning at and under geometric partitioning, on Tiny ImageNet with 10 clients and a ResNet-18 trained from random initialization or ImageNet weights (Table 4, settings in Appendix § 8). Geometric partitioning is the hardest protocol for every method, and it also orders the methods differently.
| Method | Random initialization: Label Dir. | Random initialization: Geometric | Random initialization: Semantic | Pretrained weights: Label Dir. | Pretrained weights: Geometric | Pretrained weights: Semantic |
| FedAvg | 32.970.09 (3) | 24.890.22 (4) | 35.140.35 (2) | 54.730.23 (3) | 42.110.16 (4) | 55.970.34 (5) |
| FedDyn | 30.920.13 (6) | 25.110.04 (2) | 33.910.08 (5) | 53.060.10 (7) | 36.410.33 (6) | 56.730.02 (3) |
| FedNova | 32.910.36 (4) | 20.037.69 (7) | 33.760.06 (6) | 54.220.05 (4) | 26.321.86 (7) | 55.720.39 (7) |
| FedProx | 33.070.09 (2) | 24.920.24 (3) | 34.980.09 (4) | 54.170.12 (5) | 42.950.08 (2) | 55.980.08 (4) |
| FedSAM | 30.160.18 (7) | 25.970.10 (1) | 32.660.26 (7) | 55.450.03 (2) | 45.660.08 (1) | 57.270.01 (2) |
| SCAFFOLD | 33.160.10 (1) | 24.503.57 (6) | 35.940.15 (1) | 55.630.45 (1) | 40.892.56 (5) | 58.370.18 (1) |
| FedLC | 32.270.16 (5) | 24.800.30 (5) | 35.090.17 (3) | 53.350.13 (6) | 42.420.14 (3) | 55.840.37 (6) |
| Spread | 3.00 | 5.94 | 3.28 | 2.57 | 19.34 | 2.65 |
Table 4. Accuracy (%) of seven aggregation methods on Tiny ImageNet with , from random and from ImageNet initializations. Settings differ from Table 3 (Appendix § 8). Mean standard deviation over three seeds, rank in parentheses. Geometric partitioning is the most challenging protocol for every method.
Semantic partitioning preserves much of the ranking that label skew produces, with Spearman correlations of 0.750 from random initialization and 0.464 from pretrained weights, whereas the ranking under geometric partitioning correlates with it at −0.571 and 0.179. FedSAM, last of the seven under label skew from random initialization, is first under geometric partitioning from either starting point, while FedNova falls to last. From pretrained weights the seven lie within 2.57 points under label skew, each within a point of its nearest neighbor, whereas under geometric partitioning they spread over 19.34 points, or 9.25 without FedNova, and SCAFFOLD, which edges FedSAM by 0.18 points under label skew, trails it by 4.77. From random initialization the spread widens from 3.00 to 5.94 points only because of FedNova, without which it narrows to 1.47.
Comparison to Naturally Federated Datasets
We next ask how the magnitude of heterogeneity induced by representation-space partitioning compares against that naturally present in some naturally multi-centric/federated datasets. To do this we compare the feature space and label space heterogeneity induced by in naturally federated datasets with the heterogeneity originally present, as measured by JSD, MMD, and -NN consistency, which is the average fraction of nearest neighbors in feature space belonging to the same client, measuring local clustering structure in feature space [11]. We choose Fed-ISIC2019, a skin lesion classification dataset with 6 clients [38], FEMNIST, a handwriting dataset of digits and letters separated by writer with 3,550 users [7], and Office-Caltech10 [15], a domain generalization dataset of 4 distinct domains (Amazon, Caltech, DSLR, and Webcam) with one client per domain, and compare their original client partitions against geometric partitioning under with the same number of clients. Table 5 shows that geometric partitioning often matches or exceeds the natural client split in feature space. On FEMNIST and Office-Caltech10, geometric partitioning produces substantially larger MMD and -NN client consistency than the natural partition. On Fed-ISIC2019, the natural split exhibits higher label-space divergence, while geometric partitioning produces larger feature-space separation. This distinction is important: real federations can be heterogeneous for different reasons, and the proposed framework makes the choice of representation explicit rather than reducing all heterogeneity to label imbalance.
| Dataset | Partition | Label JSD | Feature MMD | -NN consistency |
| Fed-ISIC2019 | Natural | 0.115 | 0.027 | 0.260 |
| Geometric | 0.028 | 0.192 | 0.530 | |
| Office-Caltech10 | Natural | 0.010 | 0.016 | 0.446 |
| Geometric | 0.684 | 0.334 | 0.982 | |
| FEMNIST | Natural | 0.130 | 0.043 | 0.127 |
| Geometric | 0.643 | 0.465 | 0.654 |
Table 5. Natural versus simulated heterogeneity. For each naturally federated dataset, we compare the original client split to a geometric partition with the same number of clients. Higher values indicate greater heterogeneity.
Ablating the Assignment Mechanism in Semantic Partitioning
To validate that the VLM-extracted criterion labels capture meaningful axes of variation and to examine which criteria contribute most to heterogeneity, we conduct a pure-criterion ablation. For each criterion and dataset, we create clients by deterministically assigning all samples sharing the same criterion value to the same client, bypassing the stochastic Dirichlet assignment entirely. This represents the maximum heterogeneity achievable along a single semantic axis and isolates the contribution of each criterion independently of the assignment mechanism. Table 6 (Appendix § 10) reports feature-space heterogeneity under these pure-criterion splits across all four datasets. A consistent hierarchy emerges: criteria describing what appears in an image (main object, scene, cuisine) produce the largest distributional divergence, while criteria describing how it appears (lighting, color, size) produce substantially less. This ordering is consistent across metrics and aligns with the intuition that content-level properties carry more mutual information with the task label than stylistic ones, as discussed in § 2.4. To test whether the feature-space hierarchy translates into downstream optimization difficulty, we train FedAvg on Tiny ImageNet under each pure-criterion split. The ranking is preserved exactly: main_object, the strongest criterion in feature space, yields the lowest accuracy (67.58%), a degradation comparable to geometric partitioning at (67.41%, Table 3), while time, the weakest, degrades accuracy by only 2.47 points to 72.06%. MMD and accuracy degradation across the five criteria are perfectly correlated, consistent with criteria with higher producing more consequential heterogeneity (§ 2.4). Validating the pipeline. On 10,531 Clevr-4 images [48], whose shape, texture, color, and count taxonomies are equally valid, the label of each criterion agrees with its own taxonomy far more than with the others, with NMI from 0.297 for count to 0.860 for shape, against 0.012 on average off the diagonal. Our Food101 cuisine labels reach an AMI of 0.503 with the human labels of Food-4c [33] (Appendix § 12).
| Dataset | Criterion | Clients | FID ↑ | MMD ↑ | NN sep. ↑ | Centroid ↑ |
| Tiny ImageNet | main object | 7 | 115.3 | 0.0368 | 0.840 | 13.7 |
| location | 5 | 73.2 | 0.0237 | 0.781 | 10.9 | |
| size | 4 | 60.2 | 0.0160 | 0.621 | 8.6 | |
| color | 7 | 41.3 | 0.0092 | 0.482 | 7.0 | |
| time | 4 | 29.8 | 0.0073 | 0.583 | 6.2 | |
| ADE20K | scene | 4 | 122.1 | 0.0396 | 0.853 | 14.2 |
| main object | 5 | 118.4 | 0.0275 | 0.761 | 11.6 | |
| camera | 5 | 111.2 | 0.0259 | 0.567 | 11.0 | |
| color | 7 | 165.0 | 0.0197 | 0.632 | 10.4 | |
| size | 4 | 47.7 | 0.0093 | 0.538 | 6.9 | |
| light | 3 | 17.8 | 0.0028 | 0.465 | 3.8 | |
| PASCAL VOC | main object | 6 | 228.5 | 0.0358 | 0.851 | 14.1 |
| location | 3 | 97.8 | 0.0182 | 0.729 | 9.8 | |
| size | 4 | 90.0 | 0.0131 | 0.515 | 8.0 | |
| color | 6 | 89.1 | 0.0069 | 0.329 | 6.3 | |
| light | 4 | 57.9 | 0.0035 | 0.339 | 4.5 | |
| Food101 | cuisine | 5 | 34.7 | 0.0616 | 0.764 | 16.5 |
| preparation | 5 | 42.4 | 0.0584 | 0.740 | 16.1 | |
| composition | 4 | 34.5 | 0.0274 | 0.582 | 11.3 | |
| texture | 4 | 22.5 | 0.0231 | 0.629 | 10.0 | |
| formality | 3 | 16.9 | 0.0212 | 0.733 | 9.8 |
Table 6. Pure-criterion ablation for semantic partitioning. For each criterion, one client is created per criterion value and all samples with that value are assigned to the corresponding client. Higher values indicate stronger feature-space separation.
What Geometric Partitioning Separates
Geometric partitioning requires no criteria, leaving open what its clients differ in. Against the pure-criterion partitions, they follow content far more than appearance, with a mean AMI at of 0.377 for main object, scene, and cuisine against 0.060 for lighting, color, camera, and size (Appendix § 13).
The content that a partition follows is set by the representation. Removing the top principal components before clustering shifts the partition among the properties that the encoder captures (Figure 1). For DINOv2 it raises the NMI with texture from 0.035 at to 0.428 at while that with shape falls from 0.928 to 0.186, but leaves color below 0.012, and for MAE it raises the NMI with color from 0.073 to 0.496 at , far beyond the variation across -means seeds. Encoder pretraining thus fixes what a representation can express while the projection fixes what it does express, and controls the axis of heterogeneity as controls its severity.

Figure 1. NMI between -means partitions () of Clevr-4 and each of its taxonomies after removing the top principal components of DINOv2 (left) or MAE (right) embeddings. The -means seed standard deviation at is at most 0.021 (DINOv2, three seeds). No partition collapses, since every client holds at least 1.2% of the images, and the coefficient of variation of client sizes ranges from 0.22 to 0.37 for MAE and from 0.27 to 0.61 for DINOv2.
Discussion
Feature-space and label-space heterogeneity are not interchangeable. Label Dirichlet partitioning at produces high label divergence on classification datasets but comparatively modest feature-space divergence: on Tiny ImageNet with , it achieves JSD 0.572 but only MMD , whereas geometric partitioning reaches JSD 0.655 alongside MMD . This indicates that label-skew partitions create clients that differ in which classes they hold but not in how they look, limiting the diversity of optimization challenges they present. Evaluation conducted solely under label skew therefore exercises a single axis of heterogeneity, and § 3.3 shows that heterogeneity along a different axis can change which aggregation method appears best.
Not all heterogeneity is equally consequential for federated optimization. The criterion ablation (§ 3.5) shows that the relationship between distributional divergence and FL difficulty is mediated by the mutual information between the heterogeneity axis and the task, the main_object criterion on Tiny ImageNet costing 6.95 points of accuracy against the random baseline while time costs 2.47. Effective heterogeneity simulation therefore requires partitioning along task-relevant axes, not merely maximizing distributional distance.
Dense prediction tasks are underserved by current simulation practice. On ADE20K and PASCAL VOC, where label Dirichlet partitioning is not directly applicable, random splitting has been the only available baseline. Random splits leave these tasks untested under heterogeneity, and geometric partitioning degrades mIoU on ADE20K by up to 7.88 points and mAP@50 on PASCAL VOC by up to 9.91 points relative to random. Given the growing importance of segmentation and detection in deployed FL systems, this gap represents a blind spot in current evaluation methodology that the framework directly addresses.
Geometric and semantic partitioning occupy complementary roles. Geometric partitioning produces the most challenging FL setting in most configurations and requires no design decisions beyond encoder selection, making it a strong default for stress-testing FL algorithms. Semantic partitioning trades peak heterogeneity for interpretability: the parameter provides continuous control, and the criterion-level ablation allows practitioners to understand and select the axes along which their federation is heterogeneous. The choice between the two depends on whether the goal is maximal difficulty or controlled, explainable variation. Semantic partitioning thus extends the prevailing protocol to tasks it cannot reach, while geometric partitioning creates a regime in which aggregation methods rank differently than under label skew (Appendix § 14 gives practical guidance).
Related Work
Federated dataset simulation is contemporaneous with FL itself, as the original FedAvg experiments constructed virtual non-IID clients by assigning each client a number of label shards [36]. Subsequent work by Hsu et al. [22] systematized and popularized the Dirichlet label-skew allocation first introduced in Yurochkin et al. [55], constructing a tunable protocol where the Dirichlet concentration acts as a continuous control on client heterogeneity. While this remains the dominant synthetic partitioning protocol in empirical FL [46, 23], it is limited by construction to representing client heterogeneity through variations in label marginals and therefore assumes single, sample-level class labels. Attempts to simulate feature heterogeneity instead have largely remained at the level of input perturbations, including client-specific Gaussian noise [30], image rotations [14], corruption-based perturbations [13], affine transformations [44], and Dirichlet allocation over binned feature values [18]. These methods are generally not guaranteed to affect task-relevant representations, making their induced heterogeneity difficult to interpret, measure, or transfer across tasks and label formats.
More recently, Borazjani et al. [5] explore the use of learned embedding spaces for partitioning data into non-IID clients. Their procedure fine-tunes a task-specific model, extracts latent representations, clusters them with -means, and treats the resulting clusters as pseudo-labels for a standard Dirichlet split. They demonstrate that representation-aware heterogeneity can expose failure modes missed by class-based partitions across multiple aggregation methods. However, the number of pseudo-labels is hard to interpret, and this method relies on task-specific training, making the resulting partitions difficult to standardize across studies. In contrast, we formulate federated dataset simulation as a decomposition into a representation function and an assignment mechanism , enabling label-free simulation in both continuous and semantic representation spaces and supporting evaluation through feature-space, label-space, and downstream FL metrics.
Conclusion
We introduce Federated Dataset Simulation (FDS), a framework that decomposes heterogeneity simulation into a representation function and an assignment mechanism , unifying existing label-based approaches with novel feature-level and semantic-level partitioning strategies. Through two concrete instantiations and an accompanying evaluation suite, we provide the community with a comprehensive benchmark for controlled heterogeneity research. The decomposition is inherently extensible with new representation functions and assignment mechanisms.
Broader impact. FDS lets privacy-preserving FL algorithms be stress-tested under controllable heterogeneity along several dimensions, for dense prediction as well as classification, without access to naturally federated data. The open-source release is intended to lower the barrier for FL researchers to move beyond label-skew-only evaluation.
Limitations. The current evaluation is restricted to computer vision tasks but extending the framework to text, tabular, and multi-modal data is a natural direction for future work. Greater feature-space heterogeneity and greater FedAvg degradation do not by themselves make a benchmark more realistic or more useful, and the framework offers controllable heterogeneity along several dimensions that need not resemble any particular deployment. Our evidence on what geometric partitions separate covers content and scene but not acquisition factors such as device, sensor, and compression, which would require a representation function sensitive to them. The semantic partitioning pipeline depends on the quality and consistency of VLM and LLM outputs, which may introduce noise, and it is weakest on enumeration criteria, with an NMI of 0.297 for count against 0.52 to 0.86 for the other Clever-4 criteria, a gap we attribute to the weakness of current VLMs at counting.
Acknowledgments and Disclosure of Funding
This work was supported by free cloud compute credits that RunPod provided to Giacomo Tarroni and by the Hyperion HPC cluster of City St George’s, University of London. Vasilis Siomos is a full-time employee of Ultromics and Jonathan Passerat-Palmbach is employed by Flashbots, and neither company had any involvement in this work. The other authors declare no competing interests.
The fds package
fds, available at https://github.com/siomvas/fds, is a Python package for controlled federated learning data simulation, partition evaluation, and FL training. It was designed to make the client-partitioning process explicit, reproducible, and measurable, while also providing the training infrastructure required to study how different forms of data heterogeneity affect FL convergence and final model quality.
Core Capabilities
Federated data simulation:
fdsgenerates client-level datasets for FL experiments under IID, label-skewed, embedding-based, and VLM-informed heterogeneous settings.Reusable partition artifacts: Client splits are serialized as durable artifacts, allowing the same partition to be inspected, scored, reused for training, and evaluated post hoc.
Multi-task support: The package supports image classification, semantic segmentation, and object detection workflows through task-specific data modules, models, training loops, and evaluation metrics.
End-to-end FL execution: Beyond data splitting,
fdscan launch full federated training jobs, construct client dataloaders, instantiate models, aggregate client updates, checkpoint global models, and evaluate final models.Experiment analysis: The package includes tools for quantifying partition heterogeneity, summarizing experiment grids, producing reports, and comparing convergence behavior across partitioning methods.
Datasets and Tasks
Datasets are normalized into sample dictionaries. Classification samples contain image and label fields, segmentation samples contain image and mask references, and detection samples contain image annotations such as boxes, labels, and image identifiers. Federated partitions are represented as mappings from client identifiers to sample indices. This decouples data partitioning from model training and makes each partition independently reproducible. The package supports shared global validation, client-local validation, and no-validation training. This allows experiments to distinguish between during-training validation and post-hoc evaluation of the final global model. Splitters are seed-controlled, enabling repeated generation of the same client partitions when the same configuration is used. Saved split files can be reused across model architectures, FL optimizers, or evaluation scripts without regenerating the underlying client assignments.
The package includes an easily extensible dataset factory, which currently supports classification datasets such as Tiny-ImageNet and Food101, segmentation datasets such as ADE20K, and objection detection datasets such as Pascal VOC and BDD100K. Dataset modules apply appropriate preprocessing for each task, including classification image normalization and augmentation, segmentation image-mask transforms, and detection-specific annotation handling. The same data-loading machinery can support centralized baselines and federated client-specific training. fds also includes a model factory that instantiates classification, segmentation, and detection models from supported providers timm and torchvision, transformers and segmentation-models-pytorch backbones.
Partitioning Methods
Random splitting. The random splitter provides IID or near-IID baselines by distributing samples approximately evenly across clients.
Label-Dirichlet splitting. The Dirichlet splitter creates label-skewed non-IID clients. The concentration parameter controls the severity of the label imbalance, with smaller values producing more heterogeneous client label distributions.
Embedding-based splitting. Embedding-based partitioning constructs clients according to similarity in feature space rather than according to labels alone.
VLM-based multivariate splitting. The multivariate splitter uses semantic attributes derived from vision-language annotations, allowing client heterogeneity to vary along interpretable axes such as object type, color, size, scene, location, lighting, or viewpoint.
Feature Extractors. The embedding pipeline supports representations from models such as DINOv2, CLIP, SigLIP, Inception-style networks, MAE-style encoders, raw-pixel projections, and lightweight autoencoder representations. Four registered representation functions use no pretrained weights, namely a raw-pixel projection, a sparse random projection, an untrained autoencoder, and a wavelet scattering transform. For transformer-style backbones, fds supports different token-selection and pooling strategies, such as class-token features, patch-token aggregation, and pooled multi-layer representations. Feature normalization and dimensionality reduction, including UMAP, PCA, and sliced-Wasserstein projections, can be applied before clustering, allowing the user to study how representation geometry affects client construction. Embedding partitions can be generated using methods such as K-means, HDBSCAN, KNN-based grouping, and spectral clustering.
Embedding cache. The package also includes a cache manager. Extracted features are cached, so expensive embedding computation can be reused across partitioning methods, client counts, and hyper-parameter searches.
Partition Evaluation Metrics
Label-distribution metrics: fds computes class-balance and label-divergence statistics to quantify label skew across clients.
Feature-distribution metrics: The package also evaluates heterogeneity in embedding space using distance-based and distributional measures like FID, MMD, feature-space energy distance, sliced Wasserstein distance, PCA-based multivariate KS diagnostics, kNN separability, and centroid distance.
Client balance diagnostics: The package records sample-count imbalance and related diagnostics so that pathological or undersized client splits can be identified.
Federated Training Infrastructure
fds uses NVIDIA FLARE [45] to launch and coordinate federated training jobs. Experiments are configured using structured Hydra configs that specify the dataset, partition file, model, optimizer, scheduler, FL method, client count, number of local epochs, validation policy, and output directory. Each client reconstructs its local dataloader from the saved split file and trains on its assigned samples for the configured number of local epochs. The server saves global model states during training, enabling later post-hoc evaluation or offline continuation. Runs can emit logs, manifests, TensorBoard metrics, validation summaries, and model checkpoints for downstream analysis. The comparison of aggregation methods in § 3.3 instead runs the implementations of the fluke library [41] through an adapter in fds (Appendix § 8).
Released Artifacts
The release includes the precomputed criterion labels of the four datasets, the prompts and label sets of Appendix § 9, and the artifacts and configurations of the Clevr-4 and Food-4c experiments of § 3.5.
Hyperparameter details
All federated learning experiments used FedAvg [36] with full client participation in every communication round. We train for one local epoch per round using mixed-precision training and resumed optimizer state across rounds. Experiments were trained for 100 communication rounds. Validation was performed on the global validation set during training.
For Tiny-ImageNet, we used an ImageNet-pretrained timm [52] ResNet-18 classifier with 200 output classes, cross-entropy loss, batch size 32, SGD with learning rate 0.01, momentum 0.9, weight decay 5e-6, and a cosine learning-rate schedule with minimum learning rate 1e-4. For Food101, we used 101 classes, cross-entropy loss, batch size 64, SGD with learning rate 0.01, momentum 0.9, weight decay 0, and no scheduler. For Pascal VOC, we used a COCO-pretrained torchvision Faster R-CNN ResNet50-FPN-v2 detector with 21 classes including background, batch size 2, AdamW with learning rate 1.10e-5, weight decay 0, and no scheduler. For ADE20K, we used a COCO/VOC-pretrained DeepLabV3-ResNet50 segmentation model with 150 classes and Dice loss, batch size 2, AdamW [34] with learning rate 4.5e-5, weight decay 1e-4, and a cosine schedule.
Model hyperparameters were selected with Optuna [2] using a single-client proxy training setup and validation objectives matched to each task: accuracy for classification, mean IoU for segmentation, and [email protected] for detection. We use TPE sampling with median pruning and search over batch size, optimizer, scheduler, log-scaled learning rate, weight decay, SGD momentum, and the pretrained flag for classification backbones. Separately, embedding-based partition selection was tuned with Optuna performing a full grid search over the embedding extractor, token/pooling strategy, normalization, dimensionality reduction using MMD-based heterogeneity scoring.
Comparison of aggregation methods. The experiment of § 3.3 uses Tiny ImageNet at its native resolution of with 10 clients, a ResNet-18 with group normalization trained from random initialization and an ImageNet-pretrained ResNet-18 with batch normalization, and 100 rounds of one local epoch with all clients participating, with a batch size of 64, SGD with a learning rate of 0.01, momentum 0.9, no weight decay, and no scheduler, and a server learning rate of 1.0. The methods are the implementations of fluke [41] (version 0.8.1), run through fds, which adapts the fluke implementations of FedDyn, FedNova, and SCAFFOLD. Each configuration is run with three seeds on the same partitions. The batch size, input resolution, weight decay, and schedule differ from those of the main Tiny ImageNet experiments, which accounts for the lower accuracies of Table 4.
Semantic Clustering Pipeline Details
The semantic partitioning pipeline follows IC|TC [28], with the key differences being modified prompts and additional guardrails.
Step 1: Criterion-Focused Image Description
Step 1 takes each image and asks a VLM to describe the image while focusing on a single criterion. The selected criterion is substituted into the dataset-specific step1_prompt.txt. The output for each image is a record containing the image path, original dataset label, generated description, and original sample index. The exact Step 1 prompt template is:
Describe the image focusing exclusively on [__CRITERION__].
Avoid proper nouns. Answer in <=80 tokens.
For example, Tiny-ImageNet main_object uses the prompt label Main Object, while Food101 cuisine uses cuisine or culinary origin.
Step 2a: Raw Criterion Label Extraction
Step 2a takes the Step 1 description and asks an LLM to produce a one-word answer for the selected criterion. For example, for Tiny-ImageNet, the exact Step 2a system prompt template is:
You will be given a description of an image. Your job is
to determine the [__CRITERION__]
within the image based on the provided description.
Please respond in a single word.
For example, given the following description:
"""
The image features a large tree in the middle of a green field,
with its branches casting a shadow on the grass. The tree
appears a willow tree, and its branches are covered in green leaves.
The sun is shining, creating a beautiful morning scene.
"""
An exemplar answer is "[__EXEMPLAR__]".
The exact Step 2a user prompt generated for each caption is:
Task: Read the image description and respond
with one word about the "{args.prompt_label}".
Image description: "{caption}"
Your response:
Step 2a writes the LLM output into the label field of each record in step2a_result.jsonl. These labels are raw, unconstrained strings and may include synonyms, spelling variants, or overly specific words.
Ontology Induction from Raw Labels
Step 2b takes all the raw labels, removes punctuation, counts label frequencies, filters low-frequency labels, and asks the LLM to cluster the remaining labels into a requested number of ontology categories. The exact Step 2b system prompt template, shared across datasets, is:
You will be provided a list of [__LEN__] [__CRITERION__]
and the number of occurences in a given dataset.
Your job is to cluster [__LEN__] words into
[__NUM_CLASSES_CLUSTER__] categories.
Provide your answer as a list of [__NUM_CLASSES_CLUSTER__] words,
each word representing a [__CRITERION__].
You must provide your answer in the following format
"Answer {index}: {object}", where {index} is the
index of the category and {object} is the object name
representing the category. For example, if you think the first category
is "object", then you should provide your answer as "Answer 1: object".
When categorizing classes, consider the following criteria:
1. Merge clusters with similar meanings.
2. Each cluster represents a general [__CRITERION__]
and should not be too specific.
Also, please provide a reason you chose the word for each category.
You can provide your reason in the following format
"Reason {index}: {reason}", where {index} is the index of the
category and {reason} is the reason you chose the word for the category.
The exact Step 2b user prompt generated by step2b.format_prompt is:
Task: Cluster these following list of labels into
{num_of_class} categories.
List of labels: {labels}
Your response:
Final Constrained Classification
The final stage reloads the Step 1 image descriptions and the Step 2b ontology labels, then asks the LLM to assign each image description to exactly one ontology label. This is the annotation file consumed by the VLM partition splitter. The exact Step 3 system prompt template is:
You are a classification system.
Your task: Based on the image description, determine the most
appropriate category that best classifies the [__CRITERION__] in the image.
You must choose a category from this list: [__CLASSES__]
IMPORTANT RULES:
- Output only the category name
No explanations or extra text
Choose the most specific matching category
If ambiguous, select the closest match based on dominant feature, no explanations needed
If multiple categories could apply, choose the most dominant one
Format: CategoryName (exactly as provided in options)
The exact Step 3 user prompt generated by step3.format_prompt is:
Task: Based on the image description, determine the category for
the {args.prompt_label} in the image.
You must choose one option from this list: {class_list}
Image description: "{caption}".
Base your decision on the primary/dominant characteristic.
If a situation arises where nothing is allocated,
please assign it to the object that has the closest resemblance.
Output only the category name.
Your response:
Selected Criteria
Tiny-ImageNet: main_object, size, time, color, location.
Food101: cuisine, preparation, composition, texture, formality.
ADE20K: main_object, scene, color, light, size, camera.
PASCAL: main_object, location, color, light, size.
Criterion Prompt Labels, Exemplars, and Requested Ontology Sizes
| Dataset | Criterion | Prompt label | Exemplar / requested |
| tiny | main_object | Main Object | Tree / 7 |
| tiny | size | Size and Scale of the main object | Large / 4 |
| tiny | time | Time of day | Morning / 4 |
| tiny | color | Dominant Color | Green / 7 |
| tiny | location | Location | Field / 5 |
| food101 | cuisine | cuisine or culinary origin | Italian / 5 |
| food101 | preparation | cooking or preparation method | fried / 5 |
| food101 | composition | arrangement and structure of food components | mixed / 4 |
| food101 | texture | dominant surface texture | crispy / 4 |
| food101 | formality | plating style and presentation formality | formal / 3 |
| ade20k | main_object | Main Object | Tree / 8 |
| ade20k | scene | Location | Field / 4 |
| ade20k | color | Dominant Color | Green / 7 |
| ade20k | light | Lighting Intensity | Bright / 4 |
| ade20k | size | Size of the main object | Large / 5 |
| ade20k | camera | Camera Perspective | Wide-shot / 5 |
| PascalVOC | main_object | Main Object | Tree / 5 |
| PascalVOC | location | Location | Field / 4 |
| PascalVOC | color | Dominant Color | Green / 6 |
| PascalVOC | light | Lighting Intensity | Bright / 4 |
| PascalVOC | size | Size of the main object | Large / 5 |
Table 1 (PDF p. 20).
Final Label Ontology per Criterion
The following label lists are the final ontology labels extracted from the saved training-split step2b_result.txt files. These labels are the candidate classes supplied to the final constrained classification stage. Tiny-ImageNet
color Brown/Yellow, Red, Green, Blue, Black, White, Other.
location Indoors, Urban Outdoors, Nature Outdoors, Aquatic, Unknown.
main_object Human, Animal, Vehicle, Building, Food, Landscape, Inanimate Object.
size Small, Large, Medium, Expansive.
time Day, Night, Unknown, Sunset/Dusk.
Food101
composition Mixed, Layered/Stacked, Scattered/Spread, Uniform.
cuisine American, Italian, Asian, Mexican, French.
formality Casual, Elegant, Modern.
preparation Baked, Fried, Steamed, Raw, Mixed.
texture Smooth, Rough, Crispy, Soft.
ADE20K
camera Lowangle, Eyelevel, Highangle, Aerial, Wideangle.
color Brown/Beige, White, Green, Gray, Blue, Red/Orange, Black.
light Intense, Moderate, Dim.
main_object Building, Furniture, Vehicle, Person, Nature.
scene Urban Outdoors, Residential Indoors, Natural Outdoors, Office Indoors.
size Large, Small, Medium, Huge.
Pascal VOC
color Red/Orange, Blue/Purple, Green, Black, Brown/Yellow, White.
light Dim, Moderate, Bright, Intense.
location Indoors, Urban Outdoors, Nature.
main_object Animal, Person, Vehicle, Nature, Unanimated Object, Building.
size Large, Small, Medium, Huge.
Pure-Criterion Ablation
Table 6 gives the feature-space heterogeneity of the pure-criterion splits of § 3.5.
Sensitivity of Geometric Partitioning to the Representation
Geometric partitioning involves several choices beyond the encoder, and we measured how much the representation function, normalization, and dimensionality reduction each matter. Over 733 configurations at , spanning the four datasets, eight representation functions, and reduction by UMAP to 32 or 128 dimensions or no reduction, the representation function accounts for between 58.8% and 76.3% of the variance in MMD depending on the dataset. The reduction accounts for between 0.02% and 7.8% and normalization for less than 0.2%, and FID and centroid distance order the factors in the same way. UMAP raises MMD by a median factor of 1.39 on Food101 and 1.08 on Tiny ImageNet, has no effect on the two dense prediction datasets, and ranks configurations as the unreduced embeddings do, with Spearman correlations between 0.885 and 0.909.
A further sweep of 160 configurations crosses the four datasets with four encoders and ten reduction settings, including PCA to 32 and 128 dimensions and variants of the UMAP parameters. PCA never changes which encoder produces the most heterogeneous partition, although at 128 components it shifts the magnitude of MMD by +17% on ADE20K and −22% on Tiny ImageNet. DINOv2-B, selected for the main experiments by its average MMD rank, remains the encoder with the highest MMD under the UMAP settings deployed there. Other UMAP settings change the leading encoder in six arms on Tiny ImageNet and Food101, while DINOv2-B leads throughout on ADE20K and PASCAL VOC.
Semantic Pipeline: Validation, Models, and Compute
Clevr-4. Clevr-4 [48] consists of synthetic scenes that admit four equally valid taxonomies, by shape, texture, color, and count. We run the pipeline with InternVL3.5-38B and Llama 3.3-70B on 10,531 Clevr-4 images and compare the partition produced for each criterion with each of the four taxonomies (Table 7). No stage of the pipeline reads an annotation, but the texture prompt was revised once during the run, after the descriptions produced so far had been clustered and scored against the texture taxonomy. Measured by AMI, each criterion agrees with its own taxonomy far more than with any other, by factors of 277 for shape, 34 for color, 21 for count, and 8.5 for texture over the strongest other taxonomy, and the AMI off the diagonal averages 0.010. Two controls bound these values. The four annotated taxonomies are nearly independent of one another, with a mean AMI of 0.0007, and the four criterion partitions share little with one another, with a mean AMI of 0.012. Count, the one enumeration criterion, is recovered least completely, which we attribute to the weakness of current vision-language models at counting. With Qwen3-VL-8B [3] and Qwen3-8B [53] in place of the two models, every criterion still peaks on its own taxonomy, while the NMI falls from 0.860 to 0.674 for shape and from 0.793 to 0.711 for color, so the capability of the models governs how completely a criterion is recovered and not which axis is recovered. This run kept the original texture prompt, so its texture NMI of 0.272 does not isolate the effect of the models.
Table 7: Clevr-4 validation of the semantic pipeline (InternVL3.5-38B and Llama 3.3-70B). (a) NMI between the partition produced for each criterion and each annotated taxonomy. (b) AMI between the four criterion partitions. (c) AMI between the four annotated taxonomies.
| (a) Criterion (rows) against taxonomy (columns), NMI | ||||
| Texture | Texture | Shape | Color | Count |
| Texture | 0.524 | 0.063 | 0.012 | 0.003 |
| Shape | 0.004 | 0.860 | 0.002 | 0.005 |
| Color | 0.025 | 0.008 | 0.793 | 0.002 |
| Count | 0.002 | 0.016 | 0.003 | 0.297 |
| (b) Criterion against criterion, AMI | ||||
| Texture | Texture | Shape | Color | Count |
| Texture | – | 0.041 | 0.019 | 0.001 |
| Shape | 0.041 | – | 0.005 | 0.008 |
| Color | 0.019 | 0.005 | – | 0.001 |
| Count | 0.001 | 0.008 | 0.001 | – |
| (c) Taxonomy against taxonomy, AMI | ||||
| Texture | Texture | Shape | Color | Count |
| Texture | – | 0.0000 | -0.0001 | 0.0001 |
| Shape | 0.0000 | – | 0.0000 | 0.0043 |
| Color | -0.0001 | 0.0000 | – | -0.0001 |
| Count | 0.0001 | 0.0043 | -0.0001 | – |
Table 7.
Food-4c. Food-4c [33] supplies human annotations of Food101 images along four criteria, cuisine among them. On 68,250 Food101 images, with the human cuisine labels coarsened to five categories, the cuisine labels of our pipeline reach an adjusted mutual information (AMI) of 0.503 with the human labels, an adjusted Rand index of 0.465, and direct agreement on 73.0% of images. Pipeline criteria that do not correspond to cuisine score far lower against the same human labels, 0.092 for preparation and 0.018 for texture, as do the pipeline’s cuisine labels against an unrelated human criterion, at 0.095. The five criteria of the pipeline are also nearly independent of one another, with a mean pairwise AMI of 0.032 and a maximum of 0.071, whereas the human taxonomies of Food-4c overlap far more, with a mean pairwise AMI of 0.360 and a maximum of 0.625.
Models. We tried the vision-language models MiniCPM-V 2.6 [54], InternVL2 [10], InternVL2.5-38B [9], InternVL3.5 at 8B and 38B parameters [51], and Qwen3-VL at 8B and 30B parameters [3], and the language models Llama 3.1-8B-Instruct and Llama 3.3-70B [16]. Larger models and newer generations performed better, and above a threshold of capability the choice of model family mattered little.
Compute. The labeling pipeline ran on two A100 GPUs with 80 GB of memory each. Labeling took about three days per criterion on the classification datasets of roughly 100,000 images and less than a day per criterion on ADE20K and PASCAL VOC, with criteria labeled concurrently. Labeling the 10,531 Clevr-4 images for all four criteria took about 16 hours and 4.5 million tokens.
Client Composition of Geometric Partitions
We compare each geometric partition of the heterogeneity tables with the pure-criterion partition of each criterion, for the four datasets and , which gives 63 combinations of dataset, criterion, and client count Table 8. The criterion labels cover every sample of each partition (100,000 on Tiny ImageNet, 75,750 on Food101, 25,574 on ADE20K, and 5,717 on PASCAL VOC). Over all datasets, the content criteria (main object, scene, and cuisine) reach a mean AMI of 0.379, 0.377, and 0.363 at , 10, and 20, against 0.056, 0.060, and 0.061 for lighting, color, camera, and size. On ADE20K at the ordering runs from scene (0.417) and main object (0.334) through camera (0.095), size (0.078), and color (0.033) to lighting (0.013), and total variation distances agree. Alignment also shifts with the number of clients, since on ADE20K alignment with scene falls from 0.492 at to 0.382 at , whereas on Food101 alignment with cuisine rises from 0.117 to 0.255.
| Dataset | Criterion | |||
| Tiny ImageNet | main objectc | 0.483 | 0.515 | 0.486 |
| sizes | 0.073 | 0.084 | 0.085 | |
| time | 0.080 | 0.081 | 0.075 | |
| colors | 0.053 | 0.059 | 0.072 | |
| location | 0.241 | 0.237 | 0.233 | |
| Food101 | cuisinec | 0.117 | 0.148 | 0.255 |
| preparation | 0.146 | 0.219 | 0.257 | |
| composition | 0.064 | 0.067 | 0.090 | |
| texture | 0.025 | 0.050 | 0.061 | |
| formality | 0.038 | 0.050 | 0.055 | |
| ADE20K | main objectc | 0.372 | 0.334 | 0.309 |
| scenec | 0.492 | 0.417 | 0.382 | |
| colors | 0.036 | 0.033 | 0.031 | |
| lights | 0.007 | 0.013 | 0.014 | |
| sizes | 0.083 | 0.078 | 0.078 | |
| cameras | 0.107 | 0.095 | 0.087 | |
| PASCAL VOC | main objectc | 0.432 | 0.472 | 0.381 |
| location | 0.111 | 0.226 | 0.190 | |
| colors | 0.041 | 0.052 | 0.060 | |
| lights | 0.013 | 0.028 | 0.032 | |
| sizes | 0.086 | 0.096 | 0.094 |
Table 8. AMI between each geometric partition and the pure-criterion partition of each criterion, for . cContent criteria. sCriteria of lighting, color, camera, and size.
Individual clients carry recognizable signatures Table 9. On Tiny ImageNet with 10 clients, one client holds vehicles at 10.6 times their global rate together with urban outdoor scenes at 4.3 times, another food at 9.8 times with indoor scenes at 2.1 times, and a third humans at 4.7 times. On ADE20K with five clients, one client consists of 96.1% residential indoor scenes against 31.7% in the dataset as a whole.
| Dataset | Client | Signature |
| Tiny ImageNet | 0 | location=Aquatic (3.66), main object=Animal (2.65), size=Small (1.79) |
| 1 | main object=Building (6.44), size=Expansive (5.70), main object=Landscape (5.33) | |
| 2 | main object=Inanimate Object (2.73), time=Unknown (1.98), time=Night (1.87) | |
| 3 | main object=Animal (3.53), location=Nature Outdoors (2.60), color=Brown/Yellow (1.97) | |
| 4 | main object=Vehicle (10.55), location=Urban Outdoors (4.34), color=White (2.01) | |
| 5 | main object=Human (4.68), size=Medium (1.86), time=Unknown (1.74) | |
| 6 | main object=Food (9.84), location=Indoors (2.06), color=Red (1.97) | |
| 7 | main object=Animal (3.62), location=Aquatic (3.24), location=Nature Outdoors (2.33) | |
| 8 | color=Black (4.75), main object=Animal (3.51), location=Nature Outdoors (2.44) | |
| 9 | main object=Animal (3.51), size=Medium (1.49), location=Indoors (1.44) | |
| Food101 | 0 | preparation=Raw (1.86), formality=Modern (1.75), formality=Elegant (1.56) |
| 1 | cuisine=Mexican (2.50), preparation=Mixed (1.86), preparation=Baked (1.38) | |
| 2 | texture=Crispy (3.31), preparation=Fried (3.01), composition=Scattered/Spread (1.68) | |
| 3 | preparation=Baked (2.29), composition=Uniform (1.84), cuisine=French (1.66) | |
| 4 | composition=Uniform (6.82), cuisine=American (3.36), preparation=Raw (2.67) | |
| 5 | preparation=Mixed (5.77), cuisine=Italian (5.52), composition=Mixed (3.84) | |
| 6 | preparation=Steamed (4.92), cuisine=French (3.37), preparation=Raw (2.28) | |
| 7 | preparation=Steamed (9.82), cuisine=Asian (4.55), composition=Mixed (2.47) | |
| 8 | cuisine=Mexican (2.12), texture=Crispy (1.76), texture=Soft (1.67) | |
| 9 | preparation=Steamed (10.50), cuisine=Asian (4.64), composition=Scattered/Spread (2.41) | |
| ADE20K | 0 | main object=Person (2.27), scene=Natural Outdoors (2.02), camera=Highangle (1.71) |
| 1 | scene=Residential Indoors (3.14), main object=Furniture (3.02), color=Red/Orange (2.65) | |
| 2 | camera=Wideangle (3.44), color=Black (2.79), main object=Furniture (2.73) | |
| 3 | scene=Residential Indoors (3.03), main object=Furniture (2.75), size=Medium (2.62) | |
| 4 | scene=Urban Outdoors (2.77), main object=Vehicle (2.26), main object=Building (2.26) | |
| 5 | main object=Nature (3.82), scene=Natural Outdoors (2.98), size=Small (1.85) | |
| 6 | scene=Office Indoors (4.15), color=Black (2.74), main object=Person (2.49) | |
| 7 | scene=Residential Indoors (2.78), camera=Wideangle (2.67), main object=Furniture (2.42) | |
| 8 | scene=Urban Outdoors (2.58), main object=Building (2.26), main object=Vehicle (1.60) | |
| 9 | camera=Wideangle (3.13), main object=Furniture (2.85), scene=Residential Indoors (2.58) | |
| PASCAL VOC | 0 | main object=Animal (2.75), size=Small (2.27), color=Brown/Yellow (1.41) |
| 1 | main object=Vehicle (2.73), location=Urban Outdoors (2.15), color=Red/Orange (1.71) | |
| 2 | main object=Vehicle (3.44), size=Huge (2.65), color=White (2.23) | |
| 3 | color=Green (2.88), location=Nature (2.69), main object=Animal (2.64) | |
| 4 | main object=Unanimated Object (3.29), main object=Person (2.36), location=Indoors (1.99) | |
| 5 | size=Huge (5.37), main object=Vehicle (3.29), location=Urban Outdoors (2.26) | |
| 6 | main object=Animal (2.88), location=Nature (2.59), color=Green (1.50) | |
| 7 | main object=Animal (2.89), location=Indoors (2.12), size=Medium (1.98) | |
| 8 | main object=Animal (2.54), location=Nature (2.44), size=Large (1.98) | |
| 9 | main object=Building (6.07), size=Huge (2.65), main object=Vehicle (2.26) |
Table 9. Client signatures of the geometric partitions at : for each client, the three criterion values with the largest lift (client rate over global rate) among values covering at least 5% of the client.
Practical Guidance
Choosing the representation. Content-level heterogeneity, which § 3.6 shows geometric partitions to follow, needs no intervention beyond the default encoder. Other axes call for attention to the biases of the encoder, since objectives that reconstruct masked pixels and objectives that align embeddings capture different properties of an image [40, 48], and to the projection, removing the top principal components with as the control. Where neither suffices, the representation functions that use no pretrained weights (Appendix § 7), or an encoder trained on the domain in question, can take the place of a general-purpose encoder.
Choosing criteria. To choose criteria, we start by listing the ways in which the images of a dataset actually differ. On Food101, candidate criteria for viewpoint, color, and lighting returned nearly degenerate labels, and none of the three appears among the final criteria (Appendix § 9). Each remaining candidate should be validated on a subset for a non-degenerate label distribution, the survivors ranked with the pure-criterion screen of § 3.5, and the final set checked for redundancy, which on Food101 is low, the five criteria sharing a mean pairwise AMI of 0.032 (Appendix § 12).
NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
Answer: [Yes]
Justification: We substantiate the presented claims through our framework in Section 2 and an extensive empirical analysis in Section 3. The code and artifacts are released at https://github.com/siomvas/fds.
Guidelines:
The answer [N/A] means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A [No] or [N/A] answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We discuss the limitation of our work as part of the Conclusion section.
Guidelines:
The answer [N/A] means that the paper has no limitation while the answer [No] means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate “Limitations” section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren’t acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [N/A]
Justification: This paper introduces only some definitions relating to the task of federated dataset simulation, which serve to form a common language when making this task the object of study and design. Their relevance and utility is shown empirically in the corresponding section.
Guidelines:
The answer [N/A] means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provide a best-effort explanation of all the hyper-parameters used. Along with the released code, this should be enough to replicate our results.
Guidelines:
The answer [N/A] means that the paper does not include experiments.
If the paper includes experiments, a [No] answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example (a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm. (b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully. (c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset). (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The code is released with instructions at https://github.com/siomvas/fds.
Guidelines:
The answer [N/A] means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (
https://neurips.cc/public/guides/CodeSubmissionPolicy) for more details.While we encourage the release of code and data, we understand that this might not be possible, so [No] is an acceptable answer. Papers cannot be rejected for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https://neurips.cc/public/guides/CodeSubmissionPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer) necessary to understand the results?
Answer: [Yes]
Justification: We do this in Section 3 and Appendix § 8.
Guidelines:
The answer [N/A] means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: The reason we do not run multiple seeds is that the computational burden is too high, and cannot be justified. After preliminary runs established the performance variance across seed was much lower compared to the gap between partitioning methods, we were satisfied that the insights from a single seed were correct. This can be attributed to our use of pre-trained models for initialization, which is known to greatly stabilize performance in FL optimization. The comparison of aggregation methods in § 3.3 reports the mean and standard deviation over three seeds.
Guidelines:
The answer [N/A] means that the paper does not include experiments.
The authors should answer [Yes] if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g., negative error rates).
If error bars are reported in tables or plots, the authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer: [Yes]
Justification: Appendix F reports the hardware and time of the semantic labeling pipeline. For the remaining experiments we provide part of this information, since collecting all of it would require estimating it post hoc.
Guidelines:
The answer [N/A] means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn’t make it into the paper).
Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines?
Answer: [Yes]
Justification: We have read the NeurIPS Code of Ethics and verified the submission conforms with it.
Guidelines:
The answer [N/A] means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer [No], they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).
Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We briefly discuss the potential positive impact of better FL algorithm in the conclusion.
Guidelines:
The answer [N/A] means that there is no societal impact of the work performed.
If the authors answer [N/A] or [No], they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate Deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pre-trained language models, image generators, or scraped datasets)?
Answer: [N/A]
Justification: This is not applicable to our paper.
Guidelines:
The answer [N/A] means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.
Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have cited the original owners of assets we have used, and have checked that the corresponding licenses allow the assets to be used for research purposes.
Guidelines:
The answer [N/A] means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset’s creators.
New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes] Justification: The repository at https://github.com/siomvas/fds documents the released code and artifacts and states their license, Apache 2.0.
Guidelines:
The answer [N/A] means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [N/A]
Justification: This is not applicable to our paper.
Guidelines:
The answer [N/A] means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [N/A]
Justification: This is not applicable to our paper.
Guidelines:
The answer [N/A] means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigor, or originality of the research, declaration is not required.
Answer: [Yes]
Justification: Use of an LLM is part of our semantic partitioning method. It is documented as a part of the pipeline, we reference the used model, which is open-source, and provide the exact prompts in the appendix.
Guidelines:
The answer [N/A] means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy in the NeurIPS handbook for what should or should not be described.
References
- [2]Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pages 2623–2631, 2019.arxiv.org/abs/1907.10902
- [3]Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, et al. Qwen3-VL technical report. arXiv preprint arXiv:2511.21631, 2025.arxiv.org/abs/2511.21631
- [4]Randall Balestriero, Mark Ibrahim, Vlad Sobal, Ari Morcos, Shashank Shekhar, Tom Goldstein, Florian Bordes, Adrien Bardes, Gregoire Mialon, Yuandong Tian, et al. A cookbook of self-supervised learning. arXiv preprint arXiv:2304.12210, 2023.arxiv.org/abs/2304.12210
- [5]Kasra Borazjani, Payam Abdisarabshali, Naji Khosravan, and Seyyedali Hosseinipour. Redefining non-iid data in federated learning for computer vision tasks: Migrating from labels to embeddings for task-specific data distributions. IEEE Transactions on Artificial Intelligence, 2026.
- [6]Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food-101–mining discriminative components with random forests. In European conference on computer vision, pages 446–461. Springer, 2014.DOI
- [7]Sebastian Caldas, Sai Meher Karthik Duddu, Peter Wu, Tian Li, Jakub Konečný, H Brendan McMahan, Virginia Smith, and Ameet Talwalkar. Leaf: A benchmark for federated settings. arXiv preprint arXiv:1812.01097, 2018.arxiv.org/abs/1812.01097
- [8]Liang-Chieh Chen, George Papandreou, Florian Schroff, and Hartwig Adam. Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv:1706.05587, 2017.arxiv.org/abs/1706.05587
- [9]Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271, 2024.arxiv.org/abs/2412.05271
- [10]Zhe Chen, Jiannan Wu, Weihan Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. InternVL: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24185–24198, 2024.
- [11]Chris Ding and Xiaofeng He. K-nearest-neighbor consistency in data clustering: incorporating local information into global optimization. In Proceedings of the 2004 ACM symposium on Applied computing, pages 584–589, 2004.DOI
- [12]Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge. International journal of computer vision, 88 (2):303–338, 2010.DOI
- [13]Xiuwen Fang, Mang Ye, and Xiyuan Yang. Robust heterogeneous federated learning under data corruption. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 4997–5007, 2023. doi: 10.1109/ICCV51070.2023.000463.DOI
- [14]Avishek Ghosh, Jichan Chung, Dong Yin, and Kannan Ramchandran. An efficient framework for clustered federated learning. Advances in neural information processing systems, 33: 19586–19597, 2020.arxiv.org/abs/2006.04088
- [15]Boqing Gong, Yuan Shi, Fei Sha, and Kristen Grauman. Geodesic flow kernel for unsupervised domain adaptation. In 2012 IEEE Conference on Computer Vision and Pattern Recognition, pages 2066–2073, 2012. doi: 10.1109/CVPR.2012.6247911.DOI
- [16]Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhari, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024.arxiv.org/abs/2407.21783
- [17]Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. The journal of machine learning research, 13(1):723–773, 2012.DOI
- [18]Daniel Mauricio Jimenez Gutierrez, Aris Anagnostopoulos, Ioannis Chatziagiannakis, and Andrea Vitaletti. Fedartml: A tool to facilitate the generation of non-iid datasets in a controlled way to support federated learning research. IEEE Access, 12:81004–81016, 2024. doi: 10.1109/ACCESS.2024.3410026.DOI
- [19]Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14, pages 630–645. Springer, 2016.arxiv.org/abs/1603.05027
- [20]Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15979–15988, 2022. doi: 10.1109/CVPR52688.2022.01553.DOI
- [21]Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local Nash equilibrium. Advances in neural information processing systems, 30, 2017.
- [22]Tzu-Ming Harry Hsu, Hang Qi, and Matthew Brown. Measuring the effects of non-identical data distribution for federated visual classification, 2019. URL https://arxiv.org/abs/1909.06335.
- [23]Hyejun Jeong, Shiqing Ma, and Amir Houmansadr. A survey on federated unlearning: Challenges and opportunities. IEEE Transactions on Big Data, pages 1–20, 2026. doi: 10.1109/TBDA.2026.3668358.
- [24]Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al. Advances and open problems in federated learning. Foundations and Trends® in Machine Learning, 14(1–2):1–210, 2021.arxiv.org/abs/1912.04977
- [25]Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. Scaffold: Stochastic controlled averaging for federated learning. In International conference on machine learning, pages 5132–5143. PMLR, 2020.arxiv.org/abs/1910.06378
- [26]Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. University of Toronto, 2009.
- [27]Krishan Kumar, Komal Sharma, Vinay Kukreja, and Abhishek Bhattacherjee. Exploring federated learning trends: a bibliometric analysis and word2vec nlp approach. Progress in Artificial Intelligence, pages 1–17, 2025.DOI
- [28]Sehyun Kwon, Jaeseung Park, Minkyu Kim, Jaewoong Cho, Ernest K Ryu, and Kangwook Lee. Image clustering conditioned on text criteria. arXiv preprint arXiv:2310.18297, 2023.arxiv.org/abs/2310.18297
- [29]Yann Le, Xuan Yang, et al. Tiny imagenet visual recognition challenge. CS 231N, 7(7):3, 2015.
- [30]Qinbin Li, Yiqun Diao, Quan Chen, and Bingsheng He. Federated learning on non-iid data silos: An experimental study. In 2022 IEEE 38th international conference on data engineering (ICDE), pages 965–978. IEEE, 2022.arxiv.org/abs/2102.02079
- [31]Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in heterogeneous networks. Proceedings of Machine learning and systems, 2:429–450, 2020.
- [32]Yanghao Li, Saining Xie, Xinlei Chen, Piotr Dollar, Kaiming He, and Ross Girshick. Benchmarking detection transfer learning with vision transformers. arXiv preprint arXiv:2111.11429, 2021.arxiv.org/abs/2111.11429
- [33]Mingxuan Liu, Zhun Zhong, Jun Li, Gianni Franchi, Subhankar Roy, and Elisa Ricci. Organizing unstructured image collections using natural language. In Findings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8907–8918, 2026.arxiv.org/abs/2410.05217
- [34]Ilya Loshchilov, Frank Hutter, et al. Fixing weight decay regularization in adam. arXiv preprint arXiv:1711.05101, 5(5):5, 2017.
- [35]Leland McInnes, John Healy, and James Melville. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426, 2018.arxiv.org/abs/1802.03426
- [36]Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pages 1273–1282. PMLR, 2017.arxiv.org/abs/1602.05629
- [37]Adam Narożniak. Federated datasets in research. Flower Blog, December 2024. URL https://flower.ai/blog/2024-12-02-federated-datasets-in-research/. Accessed: 2026-05-04.
- [38]Jean Ogier du Terrail, Samy-Safwan Ayed, Edwige Cyffers, Felix Grimberg, Chaoyang He, Regis Loeb, Paul Mangold, Tanguy Marchand, Othmane Marfoq, Erum Mushtaq, et al. Flamby: Datasets and benchmarks for cross-silo federated learning in realistic healthcare settings. Advances in Neural Information Processing Systems, 35:5315–5334, 2022.arxiv.org/abs/2210.04620
- [39]Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V. Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel HAZIZA, Francisco Massa, Alaaeldin El-Nouby, Mido Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Herve Jegou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bojanowski. DINOv2: Learning robust visual features without supervision. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL https://openreview.net/forum?id=a68SUt6zFt. Featured Certification.
- [40]Namuk Park, Wonjae Kim, Byeongho Heo, Taekyung Kim, and Sangdoo Yun. What do self-supervised vision transformers learn? In International Conference on Learning Representations, 2023.arxiv.org/abs/2305.00729
- [41]Mirko Polato. fluke: Federated learning utility framework for experimentation and research. arXiv preprint arXiv:2412.15728, 2024.arxiv.org/abs/2412.15728
- [42]Zhe Qu, Xingyu Li, Rui Duan, Yao Liu, Bo Tang, and Zhuo Lu. Generalized federated learning via sharpness aware minimization. In International Conference on Machine Learning, pages 18250–18280. PMLR, 2022.arxiv.org/abs/2206.02618
- [43]Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748–8763. PmLR, 2021.arxiv.org/abs/2103.00020
- [44]Amirhossein Reisizadeh, Farzan Farnia, Ramtin Pedarsani, and Ali Jadbabaie. Robust federated learning: The case of affine distribution shifts. Advances in neural information processing systems, 33:21554–21565, 2020.arxiv.org/abs/2006.08907
- [45]Holger R Roth, Yan Cheng, Yuhong Wen, Isaac Yang, Ziyue Xu, Yuan-Ting Hsieh, Kristopher Kersten, Ahmed Harouni, Can Zhao, Kevin Lu, et al. Nvidia flare: Federated learning from simulation to real-world, 2022. URL https://arxiv.org/abs/2210.13291.
- [46]David Solans, Mikko Heikkila, Andrea Vitaletti, Nicolas Kourtellis, Aris Anagnostopoulos, Ioannis Chatziogiannakis, et al. Non-iid data in federated learning: A survey with taxonomy, metrics, methods, frameworks and future directions. arXiv preprint arXiv:2411.12377, 2024.arxiv.org/abs/2411.12377
- [47]Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Re-thinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2818–2826, 2016.
- [48]Sagar Vaze, Andrea Vedaldi, and Andrew Zisserman. No representation rules them all in category discovery. In Advances in Neural Information Processing Systems, volume 36, pages 19962–19989, 2023.arxiv.org/abs/2311.17055
- [49]Jianyu Wang, Qinghua Liu, Hao Liang, Gauri Joshi, and H Vincent Poor. Tackling the objective inconsistency problem in heterogeneous federated optimization. Advances in neural information processing systems, 33:7611–7623, 2020.arxiv.org/abs/2007.07481
- [50]Jianyu Wang, Zachary Charles, Zheng Xu, Gauri Joshi, H. Brendan McMahan, Blaise Agüera y Arcas, Maruan Al-Shedivat, Galen Andrew, Salman Avestimehr, Katharine Daly, Deepesh Data, Suhas Diggavi, Hubert Eichner, Adwait Gadkari, Zachary Garrett, Antoniouѕ M. Girgis, Filip Hanzely, Andrew Hard, Chaoyang He, Samuel Horvath, Zhouyuan Huo, Alex Ingerman, Martin Jaggi, Tara Javidi, Peter Kairouz, Satyen Kale, Sai Praneeth Karimireddy, Jakub Konecny, Sanmi Koyejo, Tian Li, Luyang Liu, Mehryar Mohri, Hang Qi, Sashank J. Reddi, Peter Richtarik, Karan Singhal, Virginia Smith, Mahdi Soltanolkotabi, Weikang Song, Ananda Theertha Suresh, Sebastian U. Stich, Ameet Talwalkar, Hongyi Wang, Blake Woodworth, Shanshan Wu, Felix X. Yu, Honglin Yuan, Manzil Zaheer, Mi Zhang, Tong Zhang, Chunxiang Zheng, Chen Zhu, and Wennan Zhu. A field guide to federated optimization, 2021. URL https://arxiv.org/abs/2107.06917.
- [51]Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, et al. Internvl3. 5: Advancing open-source multimodal models in versatility, reasoning, and efficiency. arXiv preprint arXiv:2508.18265, 2025.arxiv.org/abs/2508.18265
- [52]Ross Wightman. Pytorch image models. https://github.com/rwightman/pytorch-image-models, 2019.
- [53]An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengchen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025.arxiv.org/abs/2505.09388
- [54]Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al. MiniCPM-V: A GPT-4V level MLLM on your phone. arXiv preprint arXiv:2408.01800, 2024.arxiv.org/abs/2408.01800
- [55]Mikhail Yurochkin, Mayank Agarwal, Soumya Ghosh, Kristjan Greenewald, Nghia Hoang, and Yasaman Khazaeni. Bayesian nonparametric federated learning of neural networks. In International conference on machine learning, pages 7252–7261. PMLR, 2019.arxiv.org/abs/1905.12022
- [56]Maximilian Zenk, Ujjwal Baid, Sarthak Pati, Akis Linardos, Brandon Edwards, Micah Sheller, Patrick Foley, Alejandro Aristizabal, David Zimmerer, Alexey Gruzdev, et al. Towards fair decentralized benchmarking of healthcare ai algorithms with the federated tumor segmentation (fets) challenge. Nature communications, 16(1):6274, 2025.
- [57]Jie Zhang, Zhiqi Li, Bo Li, Jianghe Xu, Shuang Wu, Shouhong Ding, and Chao Wu. Federated learning with label distribution skew via logits calibration. In International Conference on Machine Learning, pages 26311–26329. PMLR, 2022.arxiv.org/abs/2209.00189
- [58]Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Scene parsing through ade20k dataset. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 633–641, 2017.DOI