betaFeedback

WorksComputer Science

SCALE: Towards whole-slide counterfactual explanations in pathology

Abstract

Cancer diagnosis relies on histopathological evaluation, where tumor morphology guides disease classification and informs treatment decisions. Deep learning (DL) models excel at predicting diagnostic properties from histopathology images. Unlike typical medical images featuring small critical areas, histopathological images are textural, with relevant texture spanning the entire image; this makes it challenging to explain DL predictions by locating relevant image regions. Counterfactual explanations offer a promising alternative by asking "what changes in the image could have altered the model prediction." To achieve this, we propose SCALE, a framework for generating such counterfactual explanations for histopathological DL models. Across nine counterfactual directions spanning multiple cancer types, SCALE generates whole-slide counterfactual explanations that consistently shift predictions toward the intended target while maintaining biological feasibility. Blinded assessment by six board-certified pathologists showed higher morphological plausibility for SCALE (with a win rate of 91.7% over baselines) and promising counterfactual effectiveness (with a target consistency of 96.7%).

Introduction

Deep learning models in digital pathology can predict a wide range of diagnostic and molecular properties from whole-slide images (WSIs), including tumor type, histologic subtype and biomarker status Coudray et al. [ [1], Campanella et al. [ [2], Kather et al. [ [3]. A widely used paradigm for WSI prediction is attention-based multiple-instance learning (MIL), which aggregates information from many tissue regions (e.g., tiles) and assigns greater weight to regions that contribute strongly to the slide-level prediction Lu et al. [ [4]. These weights can be visualized as an attention map, highlighting regions emphasized by the model for its prediction. Yet these attention maps primarily indicate where predictive evidence is concentrated, without revealing what morphological features drive the decision Jamshidi Idaji et al. [ [5], Qin et al. [ [6]. Because relevant morphology can be distributed across large tissue regions or even the whole slide, localization alone often fails to provide an effective explanation of model behavior.

A more informative explanation would instead reveal how histologically plausible changes in morphology alter the model prediction. For example, if a model predicts a “negative” diagnosis for colon adenocarcinoma (COAD), conventional explanations methods that find regions supporting this prediction may end up highlighting the entire WSI, rendering the explanation meaningless. In contrast, counterfactual explanations ask the question of “which changes in the WSI would have caused the same model to predict a “positive” diagnosis. As illustrated conceptually in Supplementary Fig. 1, counterfactual1 can make such prediction-altering morphological changes explicit, revealing the tissue features the model associates with alternative pathological states. Showing whether predictions respond to pathologically plausible morphology can help determine whether model decisions rely on meaningful tissue features rather than spurious visual cues, providing a stronger basis for assessing model trustworthiness Zigutyte et al. [2026]. Realizing this potential at the whole-slide level, however, requires counterfactual methods that can not only identify prediction-relevant morphology, but also generate plausible morphological changes at the large spatial scale over which that evidence is distributed.

Existing interpretability and counterfactual methods satisfy only subsets of these requirements. Representation- and concept-based methods can recover interpretable morphological patterns associated with model predictions Lu et al. [2021], Qin et al. [2026], but remain largely associational and do not show how the corresponding tissue morphology would need to change to produce a different outcome. Slide-level counterfactual approaches can manipulate tissue instances or representations Kaczmarzyk et al. [2024], but are not generative and therefore cannot synthesize the corresponding morphological transformations. Generative counterfactual methods can instead produce prediction-altering histopathology images Zigutyte et al. [2026], Muñoz et al. [2026], but typically operate on localized image regions, e.g., on individual small tiles, and do not address how such interventions can be coherently propagated across larger tissue extents. Thus, generating counterfactuals that are simultaneously morphologically interpretable, spatially targeted, and scalable to the whole-slide level remains an open challenge.

Developing such a framework requires addressing three coupled technical problems: what morphology to change, where to apply the change, and how to propagate it across the WSI. First, a counterfactual intervention must specify what morphology to change in an interpretable manner rather than perturbing arbitrary features in a high-dimensional representation. Second, because WSIs contain extensive and heterogeneous tissue, the framework must determine where an intervention should be applied so that the edited regions are relevant to the model prediction. Third, the gigapixel scale of WSIs makes whole-slide generation impractical in a single pass, requiring a strategy for determining how localized morphological changes can be coherently propagated across larger tissue regions while preserving continuity between neighboring edits.

To address these challenges, we introduce Scalable Counterfactuals through Attention-guided Latent Editing in Pathology (SCALE), a pathology-grounded framework that unifies what morphology to change, where to intervene, and how to propagate counterfactual edits across tissue, enabling interpretable counterfactual generation that scales from local regions to the whole-slide level (details in Figure 1 and Sec. 2.1). Across nine counterfactual directions spanning six cancer types and multiple biological endpoints, including normal-to-tumor transitions, human papillomavirus (HPV)-associated morphology, and prostate cancer growth patterns, SCALE produces coherent, concept-aligned morphological transformations localized tissue regions to whole-slide images, while consistently moving model predictions toward the intended targets. Blinded assessment by six pathologists further evaluates the relative morphological plausibility and target consistency of the generated counterfactuals, complementing classifier-based evaluation with expert assessment of tissue morphology. Together, these results establish SCALE as a unified framework for interpreting whole-slide pathology models through explicit morphological counterfactuals.

Overview of SCALE, showing feature extraction, attention-based multiple instance learning, and concept-guided counterfactual generation

Figure 1. Overview of SCALE. a, Feature extraction. Whole-slide images are partitioned into tiles and encoded by a pretrained tile encoder to obtain tile-level feature embeddings. Each slide is represented as a feature matrix X∈RN×DX \in\mathbb{R}^{N \times D}, where NN denotes the number of tiles and DD denotes the feature dimension. b, Attention-based multiple instance learning. Tile-level feature embeddings are aggregated using an attention-based MIL model to produce slide-level predictions. The model assigns attention weights to individual tiles, highlighting regions that contribute most to the prediction and enabling attention-guided region selection for generating counterfactual explanations. c, Concept-guided counterfactual generation. An SAE is trained on large-scale TCGA data to discover a reusable concept bank from pathology feature representations, with each concept grounded by prototype examples. Concept-class association identifies task-relevant concepts, and a target concept cj∗c_{j^*} is selected for intervention. The original feature xx is steered in the SAE latent space to obtain a modified feature x′x'; a diffusion model is then conditioned on x′x' to generate region-level (e.g., tile-level) counterfactual explanations. Progressive editing integrates these local edits across the slide to produce a coherent slide-level counterfactual with the desired prediction shift.

Results

Overview of SCALE

Figure 1 summarizes SCALE as a closed-loop counterfactual explanation framework for identifying and generating morphological changes that alter whole-slide model predictions. It consists of three main stages.

Each WSI is first partitioned into tiles, which are encoded using a pretrained tile encoder to obtain tile-level feature embeddings, as shown in Figure 1a. A slide is represented by a feature matrix X∈RN×DX \in\mathbb{R}^{N \times D}, where NN is the number of tiles and DD is the feature dimension.

These tile-level embeddings are then aggregated using an attention-based MIL model to produce slide-level predictions, as shown in Figure 1b. The learned attention weights quantify the contribution of individual tiles to the prediction and are used to identify salient regions for subsequent counterfactual explanation.

Most importantly, as shown in Figure 1c, we introduce a concept-guided counterfactual generation framework built on an SAE trained on large-scale TCGA pathology data to learn a reusable bank of concepts from pathology feature representations, with each concept grounded using prototype examples. Concept-class associations are then used to identify task-relevant concepts and select a target concept cj∗c_{j^*} for intervention. Given an original tile feature xx, we steer its representation in the SAE latent space toward the target concept to obtain a modified feature x′x'. A diffusion model conditioned on x′x' subsequently generates a tile-level counterfactual image corresponding to the desired concept intervention. Finally, progressive editing integrates these local (tile-level) counterfactual edits across salient regions of the slide, yielding a coherent slide-level counterfactual that induces the desired shift in the model prediction.

The generated counterfactual image (consisting of multiple tiles) is returned to the same downstream model to measure whether the morphological intervention shifts the prediction toward the intended target, closing the counterfactual loop (Figure 1c).

Below, we first establish that SCALE’s learned concepts provide interpretable morphological patterns for intervention, test whether SCALE’s interventions alter model predictions and where they are most effective, and then evaluate the pathological validity and spatial scalability of SCALE’s generated counterfactuals. Further implementation details for each component of the workflow are provided in the Methods section.

Pan-cancer foundation-model representations yield interpretable, reusable morphological concepts

Applying SAE decomposition to pan-cancer pathology foundation-model representations yielded a reusable set of visually interpretable morphological concepts. To capture broad morphological diversity, we trained the SAE on foundation-model representations Chen et al. [ [11] from 141.1 million tissue tiles across 10,484 WSIs and 8,641 patients in 32 TCGA cohorts and 25 anatomical sites Hoadley et al. [ [12] (Figure 2a). Tile contributions were broadly distributed across organ systems and cancer types, providing a diverse representation space from which shared morphological concepts could be identified (Figure 2b provides more detailed statistics).

Figure 2. Pan-cancer concept discovery and class-associated morphology. a, Sparse autoencoder (SAE) training data comprise 141.1 million tissue tiles from 10,484 whole-slide images and 8,641 patients across 32 The Cancer Genome Atlas (TCGA) cohorts and 25 anatomical sites. Tiles measure 256×256256 \times256 pixels at 20×20\times magnification or 0.5 μm0.5\,\mu\mathrm{m} per pixel. b, Tile contributions by cohort, grouped by anatomical system; labels report counts in millions. c, The SAE maps a dense tile feature x∈R1536x \in\mathbb{R}^{1536} to an overcomplete sparse concept vector z∈R12288z \in\mathbb{R}^{12288}. Colored and gray entries schematically indicate nonzero and zero activations, respectively. d, Spatial concept map of an example tissue region. Each tile is colored by its highest-activation SAE concept; different colors denote distinct concepts. e, Representative top-activating tiles for concepts C12283C_{12283}, C2957C_{2957} and C7920C_{7920}, illustrating trabecular hepatic architecture, cytoplasmic vacuolization and fascicular or striated muscle, respectively. Subscripts denote SAE concept indices. f, Class-associated concepts across histologic subtype, tumor grade and molecular status: keratinization in LUSC and gland formation in LUAD; low atypia and pleomorphic nuclei in low- and high-grade KIRC; and basaloid morphology and keratinization associated with HPV+^{+} and HPV−^{-} head-and-neck squamous cell carcinoma, respectively. Galleries show representative top-activating tiles for the concept with the highest mean attention-weighted activation within each class.

The resulting sparse concepts captured recurring and spatially coherent tissue morphologies. Top-activating tiles for individual concepts showed recognizable tissue structures and cellular patterns, even though concept discovery does not rely on predefined morphological categories or tile-level concept annotations (Figure 2c,Figure 2e). Mapping dominant concept activations across WSIs further revealed spatially organized morphological domains within tissue (Figure 2d), indicating that the sparse representation recovered coherent morphological patterns from the underlying foundation-model feature space.

Importantly, the same concept space could be reused across distinct downstream prediction tasks and cancer types. Attention-weighted concept distributions differed between classes (Figure 2d), and strongly class-associated concepts exhibited distinct prototype morphologies across histologic subtype, tumor grade and viral-status settings (Figure 2f). For example, Figure 2f (left) shows that two histologic subtypes of lung cancer, lung squamous cell carcinoma (LUSC) and lung adenocarcinoma (LUAD), show different squamous and gland-forming morphologies, respectively, in their representative tiles. Similar examples include low nuclear atypia versus prominent nucleoli and pleomorphic nuclei in clear-cell renal cell carcinoma, shown in Figure 2f (middle), and basaloid versus keratinizing patterns associated with HPV status in head-and-neck squamous cell carcinoma, shown in Figure 2f (right). Together, these results show that pan-cancer foundation-model representations can be decomposed into visually grounded morphological concepts that are reusable across prediction tasks and provide interpretable targets for subsequent counterfactual intervention.

Concept-guided counterfactual explanations reveal how morphology alters model predictions

Concept-guided counterfactual explanations revealed morphological changes that systematically altered downstream model predictions across diverse pathology tasks. Unlike attention-based interpretability, which only localizes prediction-relevant regions, SCALE performs counterfactual interventions by steering the original WSI (e.g., with HPV−^{-}) toward morphology more associated with target label (e.g., HPV+^{+}) and measures how the same prediction model responds to the resulting counterfactuals. We evaluated nine counterfactual directions spanning breast, colorectal, kidney, lung, head-and-neck and prostate pathology, including normal-to-tumor transitions, bidirectional HPV-status transitions (between HPV+^{+} and HPV−^{-}), and changes between prostate growth-pattern classes. Systematic evaluation was performed primarily on 2048 × 2048-pixel tissue regions to provide a common large-region setting across tasks. For these experiments, predictions were computed from each region’s tile bag using the same task-specific MIL model, with all tiles affected by generation or blending re-encoded before evaluation.

Across nine counterfactual directions, target-class probability generally increased with the percentage of tissue tiles edited across WSIs, evaluated on 134 distinct WSIs from 101 TCGA cases (Figure 3). However, the editing extent required to induce substantial prediction shifts varied considerably across tasks. Among the normal-to-tumor transitions,

Whole-slide prediction responses to increasing counterfactual editing extent using SCALE across nine pathology tasks

Figure 3. Whole-slide prediction responses to increasing counterfactual editing extent using SCALE across nine pathology tasks. Target-class probability as a function of the percentage of tissue tiles counterfactually edited across whole-slide images (WSIs). The evaluation included n=134n = 134 distinct WSIs from 101 TCGA cases across nine counterfactual task directions. For PRAD, Gleason patterns 1–3 (GP1–3) were grouped into a single class characterized by discrete, well-formed glands based on the historical Gleason morphological classification. Panels show, from left to right and top to bottom: BRCA normal-to-tumor, COAD normal-to-tumor, KIRC normal-to-tumor, HNSCC HPV−^{-}-to-HPV+^{+}, HNSCC HPV+^{+}-to-HPV−^{-}, LUAD normal-to-tumor, PRAD GP1–3-to-GP4, PRAD GP4-to-GP5 and PRAD GP1–3-to-GP5 editing. Horizontal red dashed lines indicate a target-class probability of 0.5 as a common reference level.

COAD and KIRC showed rapid responses, with target-class probabilities approaching 1.0 after editing approximately 6% and 25% of tissue tiles, respectively. In contrast, BRCA and LUAD exhibited more gradual increases, reaching approximately 0.9 at 25% editing. HPV-status transitions also showed asymmetric responses: HNSCC HPV+^{+}-to-HPV−^{-} reached a target-class probability near 1.0 at 50% editing, whereas HPV−^{-}-to-HPV+^{+} crossed the 0.5 decision threshold only after editing 50% of tissue tiles and reached approximately 0.7 at 100% editing. For PRAD, GP4-to-GP5 editing produced a particularly rapid response, exceeding 0.9 target-class probability after editing approximately 1% of tissue tiles, while GP1–3-to-GP4 and GP1–3-to-GP5 transitions showed more gradual increases. Consistent target-directed prediction shifts were also observed in the complementary large-region evaluation using fixed editing budgets of 1–48 tiles (Supplementary Fig. 2), further demonstrating the effectiveness of SCALE across different spatial scales. Together, these findings demonstrate that increasing the spatial extent of counterfactual interventions induces target-directed prediction shifts across both whole-slide and large-region settings, although the editing extent required varies substantially across pathology tasks.

Spatial selection, steering strength and intervention extent provide complementary control

Having established target-directed prediction responses, we next examined control over intervention location, steering strength, and spatial extent. SCALE uses attention from the same MIL model whose prediction is being explained to prioritize tissue tiles for intervention. In the representative colorectal example (Figure 4a), attention-guided selection concentrated edits within high-attention, epithelium-rich tissue containing glandular structures directly relevant to the intended morphological transformation. Random selection at a matched selected-tile budget instead dispersed interventions across heterogeneous tissue compartments, including areas with relatively sparse epithelial content. This verifies that SCALE’s attention-guided selection aligns the intervention with both the model’s spatial weighting and the tissue structures where the requested morphological change could be meaningfully expressed. Consistent with this qualitative example, attention-guided selection produced larger or equal target-probability shifts than random selection at matched selected-tile budgets across all nine intervention directions, with the largest advantage in directions where target-associated tissue was spatially sparse, such as the HPV−-to-HPV+ transition (Supplementary Fig. 3).

Figure 4. Spatial selection, steering strength and intervention extent. a, Colorectal counterfactuals generated using attention-guided selection (left) or random selection (right) at a matched selected-tile budget. The inset highlights high-attention tiles selected for intervention. Red outlines indicate representative intervention areas, rather than exact affected-pixel masks. b, Steering-strength sweep in a representative kidney region, with the selected area fixed (red box). Images show the source and counterfactuals at increasing steering strengths. Plots show region-level target-class probability (left) and mean target-concept SAE activation across the indicated 16 tiles (right). c, Intervention-extent sweep in the same kidney region, with steering strength fixed and increasing percentages of valid tissue tiles selected for intervention. Plots show region-level target-class probability (left) and mean target-concept SAE activation across all 55 valid tissue tiles (right).

To distinguish the effects of local steering (intervention) strength (for each tile) and spatial extent (i.e., how many tiles are edited), we varied each aspect independently in a representative region. For the steering-strength experiment, the steering (intervention) was restricted to the fixed region indicated by the red box while only steering strength was increased. Increasing steering strength from 0% to 40% increased target-class probability from 0.021 to 0.999 and mean target-concept SAE activation from 0 to 2.29; further increasing steering strength to 100% raised target-concept activation to 6.33 while the classifier response remained near saturation (Figure 4b). Increasing the fraction of edited valid tissue (i.e., number of tiles) produced a similarly strong target-directed response: target-class probability increased from 0.013 without steering to 0.99987 at 20% editing, while mean target-concept activation increased progressively from 0 to 5.97 as the edited fraction expanded to 100% (Figure 4c). Together, these experiments separate two complementary dimensions of counterfactual control: how strongly target-associated morphology is introduced at each selected location and how broadly the intervention is distributed across tissue. SCALE therefore reveals not only what morphology can change a prediction, but also how strongly and how extensively that morphology must be represented for the prediction model to respond.

We also tested whether the selected SAE concept could be steered directly through a single latent dimension. Single-neuron steering often produced off-target changes or visible artifacts, while SCALE’s prototype-guided steering yielded more coherent target-associated morphology (Supplementary Fig. 4). This supports using the full latent prototype rather than an isolated SAE dimension for counterfactual generation.

SCALE generates target-associated counterfactual explanations from local regions to whole slides

For a counterfactual to provide a meaningful morphological explanation of model behavior, the prediction shift should correspond to a recognizable and pathologically plausible alternative tissue state rather than an arbitrary image perturbation. As illustrated in Figure 5a, SCALE produced target-directed prediction changes together with visible morphological transformations consistent with the corresponding pathological state. In COAD’s normal-to-tumor counterfactuals (Figure 5a(left)), the regular organization of colonic crypts was replaced by more crowded and irregular gland-forming epithelium, consistent with the glandular architectural abnormality characteristic of colorectal adenocarcinoma [13, 14]. In kidney tissue (Figure 5a(middle)), editing toward the tumor class introduced nests of cells with clearer cytoplasm and altered epithelial architecture characteristic of clear-cell renal cell carcinoma [15]. HPV-positive head-and-neck tissue edited toward the HPV-negative class showed a shift from predominantly basaloid morphology toward more eosinophilic, squamous-differentiated and keratinizing features, consistent with established HPV-associated morphological differences in HNSCC [16, 17, 18] (Figure 5a(right)). Importantly, these transformations remained organized at the cellular and tissue-architectural levels rather than appearing as arbitrary texture changes, supporting their interpretation as histologically plausible counterfactual alternatives and allowing them to serve as counterfactual explanations for the predictions (e.g., HPV-positive head-and-neck tissue). Pathological review indicated that these transformations remained histologically plausible and recognizable as the intended counterfactual alternatives.

Representative counterfactual explanations across pathology tasks from SCALE

Figure 5. Representative counterfactual explanations across pathology tasks from SCALE. a, Source and concept-guided counterfactual regions for colorectal (COAD) normal-to-tumor, kidney (KIRC) normal-to-tumor and HNSCC HPV+^{+}-to-HPV−^{-} editing. For PRAD, the same source region is steered toward progressively higher-grade Gleason pattern (GP) targets, illustrating controlled changes in glandular architecture. b, Colorectal normal-to-tumor counterfactuals generated while increasing the fraction of edited valid-tissue tiles from 40% to 80%. Increasing the edited fraction expands tumor-associated morphology across the tissue region, providing a spatial analogue of increasing local tumor involvement. Percentages denote the fraction of valid-tissue tiles selected for intervention. c, Representative COAD normal-to-tumor whole-slide counterfactual. Progressive editing propagates the target-associated morphological intervention across spatially distributed tissue regions while retaining the overall slide context. Red boxes indicate the enlarged region. Across all panels, counterfactual labels indicate intended intervention targets rather than definitive pathological classifications of the generated tissue.

Besides binary cases, SCALE also captured more graded morphological alternatives. In prostate tissue (Figure 5a(bottom)), steering the same source region toward progressively higher-grade growth patterns produced corresponding changes in glandular architecture consistent with established Gleason pattern (GP) distinctions [19 b] and the requested (higher-grade) target patterns. In colorectal tissue, increasing the fraction of edited valid-tissue tiles from 40% to 80% progressively expanded tumor-associated morphology across the region (Figure 5b), demonstrating control over the spatial extent of the morphological explanation. Together, these results show that SCALE explains changes in model predictions through explicit, histologically plausible morphological alternatives, grounding the counterfactual explanation in tissue changes that are recognizable in pathological terms.

To assess the morphological quality and effectiveness of the generated counterfactuals beyond classifier outputs, six board-certified pathologists evaluated 30 cases spanning HPV-status, PRAD growth-pattern, and COAD tasks in a blinded assessment, yielding 60 reader-case assessments for each of two complementary questions. First, we assessed morphological plausibility by asking reviewers to compare SCALE with the evaluated baselines and rate which generated image appeared most histologically plausible. Second, we assessed counterfactual target consistency by asking whether the SCALE-edited image or its corresponding source image was more consistent with the morphology associated with the requested target class (for example, basaloid morphology for an HPV+ target versus keratinizing morphology for an HPV− target, or architectural changes consistent with a transition from lower-grade prostate cancer growth patterns (GP1-3) toward GP4 or GP5).

For morphological plausibility, SCALE achieved a strict win rate of 91.7% (55 of 60 reader-case assessments), receiving ratings higher than both single-neuron SAE steering and independent-window editing. Neither baseline achieved a strict win (Supplementary Table 1). For counterfactual target consistency, pathologists selected SCALE counterfactual explanations over their corresponding source (original) images as more consistent with the requested target morphology in 58 of 60 reader-case assessments (96.7%; Supplementary Table 2). These assessments covered 30 unique cases, each evaluated by two pathologists. Together, these results establish that SCALE produces counterfactual explanations that expert pathologists both accept as histologically realistic and recognize as expressing the requested target morphology, the two properties a counterfactual explanation must have to be trusted as evidence about model behavior.

A central goal of SCALE is to extend localized counterfactual interventions from selected tissue regions to the spatial scale of whole-slide pathology. We therefore evaluated whether the same concept-guided, attention-directed editing strategy could produce coherent target-associated changes across a WSI while preserving the broader tissue context. In a representative COAD normal-to-tumor example, SCALE progressively targeted high-attention regions prioritized by the model and introduced tumor-associated glandular morphology without globally replacing or restructuring the slide (Figure 5c). Enlarged views showed localized transitions from organized colonic glands toward more irregular tumor-associated glandular architecture, while surrounding tissue organization remained largely preserved.

Importantly, these spatially distributed edits produced a corresponding shift in the whole-slide prediction. In the supplementary whole-slide analysis (Supplementary Fig. 5), the classifier’s predicted probability of the tumor class (target), averaged across all evaluated slides, increased from approximately 0.02 to 0.72 after only about 50 edited tiles, crossing the 0.5 decision threshold, and reached approximately 0.99 after about 140 tiles. This shows that SCALE can accumulate localized, morphologically targeted interventions across an entire WSI to induce a controlled change in the slide-level prediction. This whole-slide setting demonstrates that the SCALE framework is not restricted to isolated image patches or fixed regions, but can scale counterfactual explanations to the spatial context in which computational pathology models operate.

Progressive editing enables scalable counterfactual generation with reduced boundary artifacts

Counterfactual explanations of whole-slide predictions require morphological interventions that can extend beyond the bounded field of view of a local image generator. SCALE achieves this through progressive overlapping-window editing, which coordinates successive local generations through shared spatial context (Figure 6a,Figure 6e). This strategy allows target-associated morphology to be propagated across increasingly large tissue regions while maintaining continuity between neighboring edited and unedited areas.

Figure 6. Spatial continuity and prediction changes during progressive editing. a, Local editing uses overlapping 4×44 \times4-tile windows with a fully edited 2×22 \times2 center and a blended peripheral region (10% generated, 90% existing content). b, Source tissue and counterfactuals generated by independent stitching or progressive editing (ours). Green boxes mark zoomed-in regions. c, Mean absolute RGB change (left) and excess border discontinuity (right) across increasing selected-tile budgets (lower is better for both metrics). Lines indicate means, and shaded bands indicate the standard error of the mean across nine intervention directions (n=9n = 9). d–g, Representative HPV+^{+}-to-HPV−^{-} intervention. d, Source region, attention map and selected tiles. e, Progressive editing through overlapping windows. f, Source and resulting counterfactual. g, Target HPV−^{-} probability across editing steps (left) and source and counterfactual class probabilities (right). Target probability increases from 0.002 to 0.955; the dotted line marks the 0.5 decision threshold.

Progressive editing links these local operations across a shared spatial grid. Each local generation window operates on a 256×256256 \times256-pixel tile, and the 2048 × 2048-pixel regions used for systematic evaluation were therefore partitioned into an 8×88 \times8 conditioning grid of such tiles (8 × 256 = 2048). This grid was traversed using overlapping local generation windows (Figure 6d,Figure 6e). Each newly generated tile was incorporated into the evolving tissue image and used as spatial context for subsequent interventions, extending the morphological changes across the selected region (details on progressive editing in § 4.6).

We compared progressive editing with an independent-window baseline using matched target concepts, editing locations and generation windows. In the baseline, each window was generated independently and the outputs were assembled by hard stitching. Figure 6b shows that the independent-window baseline produced visible seams and abrupt discontinuities in tissue architecture at tile boundaries, whereas progressive editing yielded seamless, spatially coherent transitions in which generated tiles blended naturally with their neighbors and with the surrounding unedited tissue.

Averaged across nine task directions, progressive editing produced lower mean absolute RGB change and lower excess border discontinuity (defined in § 4.8) than the independent-window baseline (Figure 6c). At a selected-tile budget of 48, mean absolute RGB change was 49.39 for progressive editing versus 63.72 for the independent-window baseline, a reduction of approximately 22%. Excess border discontinuity was 3.70 for progressive editing versus 18.92 for the baseline, corresponding to an approximately 80% reduction in boundary discontinuity. These results indicate that our

SCALE’s progressive editing reduces overall pixel alteration and boundary artifacts relative to independent-window editing.

In the representative HPV+\mathrm{HPV}^{+}-to-HPV−\mathrm{HPV}^{-}-targeted intervention, attention-guided selection and progressive window traversal extended editing across the source region (Figure 6d–Figure 6f). The region-level HPV−\mathrm{HPV}^{-} probability increased from 0.002 to 0.955, crossing the decision threshold during editing (Figure 6g). This result shows that SCALE can successfully flip a region-level prediction by coordinating local edits into a much larger, visually coherent, counterfactual region, laying the foundation for whole-slide counterfactual generation (as shown in § 2.5).

Discussion

We developed SCALE, a pathology-grounded framework for explaining whole-slide model predictions through spatially controlled morphological counterfactuals. Across nine counterfactual directions spanning six cancer types and multiple pathological endpoints, SCALE generated recognizable target-associated morphological changes that consistently shifted model predictions toward the intended targets (e.g., from HPV+\mathrm{HPV}^{+} to HPV−\mathrm{HPV}^{-}), thereby counterfactually explain the original model prediction (e.g., HPV+\mathrm{HPV}^{+}). Counterfactual responses depended on both the local strength and spatial extent of the intervention, while attention from the model being explained identified tissue regions in which morphological changes were most relevant to the prediction. SCALE’s progressive editing further propagated localized interventions across large tissue regions and toward whole-slide scale while limiting unnecessary image perturbation and boundary artifacts. Together, these results demonstrate that with SCALE’s counterfactual explanations, whole-slide model behavior can be examined through explicit changes in tissue morphology rather than localization alone.

A central contribution of SCALE is to move pathology model interpretation from association toward controlled morphological intervention. Attention-based approaches can identify tissue regions associated with a prediction Lu et al. [ [4], while representation-based methods can identify interpretable morphological concepts and other cancer-related features Le et al. [ [21], Qin et al. [ [6], Su et al. [ [22]. These approaches reveal where predictive information is concentrated or which morphological patterns are associated with a model output, but do not directly test how changing those patterns alters the prediction. Slide-level counterfactual approaches Kaczmarzyk et al. [ [9] construct counterfactual slides by removing, adding or replicating existing patch embeddings, without synthesizing any pathological images; they are not generative methods and therefore cannot generate counterfactual explanations. On the other hand, alongside broader advances in diffusion guidance Wang et al. [ [23], Yang et al. [ [24], generative counterfactual methods can produce prediction-altering histopathology images Zigutyte et al. [ [8], Muñoz et al. [ [10] but remain task-specific and limited to single-tile generation. Direct manipulation of individual interpretable latent features has also been explored Kim et al. [2026]; in our setting, such isolated SAE-feature steering frequently produced off-target morphology or image corruption. SCALE closes this gap by, for the first time, unifying WSI-level counterfactual reasoning with interpretable, spatially controlled pathological generation, enabling direct tests of how specific morphological changes alter whole-slide predictions.

More importantly, morphological counterfactuals provide a direct way to assess whether model decisions are grounded in pathologically meaningful evidence. When the model’s prediction on the counterfactual image shifts in the intended direction in response to histologically plausible changes in recognizable tissue structures, the counterfactual provides evidence about the morphology the model relies on for its decision. Conversely, two kinds of outcome warrant further investigation. If the prediction shifts substantially even though the counterfactual contains no coherent change in diagnostically relevant morphology, the model may be relying on spurious visual cues (for example, staining intensity or background texture) rather than tissue architecture. If the prediction remains unchanged despite a clear, histologically appropriate morphological change, the model may not be using the features a pathologist would consider decisive. Counterfactual explanation therefore complements conventional predictive performance by providing an additional basis for evaluating how a model reaches its decisions and, ultimately, for assessing model trustworthiness. Blinded assessment by six pathologists across 30 HPV-status, PRAD growth-pattern, and COAD provided complementary support for these morphological interventions. SCALE achieved the highest plausibility win rate among the evaluated methods, and readers selected its counterfactuals over source images as more consistent with the requested target morphology in nearly all paired assessments. These findings support both the relative morphological plausibility of SCALE and its ability to introduce target-associated changes (i.e., counterfactual explanations) recognizable to pathologists.

SCALE uses a reusable morphological concept space to define these counterfactual interventions across diverse pathology tasks. Rather than learning a separate representation for each prediction problem, as also explored in medical image generation Chen et al. [2026], the same visually grounded concepts can be queried to identify target-associated morphology and reused as intervention targets across cancer types and endpoints. This makes the counterfactual images interpretable in morphological terms rather than as an arbitrary perturbation in representation space. The intervention experiments further revealed distinct aspects of model sensitivity: model responses depended both on how strongly target-associated morphology was expressed locally and on how broadly it was distributed across tissue. Attention provided a complementary spatial dimension by identifying where these concept-guided interventions were most relevant to the model prediction. Together, these capabilities allow counterfactuals to probe not only which morphology matters, but also how its expression and spatial distribution influence model behavior.

Spatial scalability is particularly important in whole-slide pathology because prediction-relevant morphology may extend across tissue regions far larger than the field of view of a local image generator. Our SCALE’s Progressive editing allowed localized morphological interventions to be propagated across neighboring regions while maintaining continuity with previously generated and surrounding tissue. Compared with independent-window generation, this strategy substantially reduced boundary discontinuities and unnecessary image alteration, preventing seams and mismatched morphology from accumulating as the counterfactual expanded. The same procedure successfully scales from systematically evaluated 2048×20482048 \times2048-pixel regions to complete WSIs. Progressive editing therefore provides the spatial mechanism needed to preserve plausible morphology as counterfactual explanations extend from local regions toward whole-slide scale.

Several limitations remain. First, SCALE enables counterfactual generation across whole-slide spatial extents, but the current counterfactual intervention remains defined at the level of tissue tiles rather than individual histological structures. This provides scalable regional control but does not yet allow direct manipulation of specific glands, cells or subcellular features; future work could incorporate finer-grained or hierarchical representations to enable more precise morphological interventions while retaining whole-slide scalability. Second, the learned concept space depends on the underlying pathology foundation model, so poorly represented morphological attributes may be difficult to isolate or manipulate. Future work could explore alternative foundation models to obtain complementary or finer-grained concept spaces.

SCALE extends whole-slide interpretation from asking where predictive evidence resides to testing which pathologically plausible tissue changes would lead the same model to a different conclusion. By making these prediction-altering morphological changes explicit across spatial scales, the framework provides a direct means to examine whether model decisions are grounded in morphologically meaningful evidence; this is an important step toward the transparent and trustworthy deployment of digital pathology models in clinical practice Kelly et al. [2019].

Methods

Dataset preprocessing and feature extraction

We used hematoxylin-and-eosin-stained whole-slide images (WSIs) from The Cancer Genome Atlas (TCGA) Weinstein et al. [2013] spanning multiple cancer types. The pan-cancer dataset was partitioned at the patient level into an SAE training set and a held-out test set. The SAE training set comprised 10,484 diagnostic WSIs from 8,641 patients across 32 TCGA cohorts and 25 anatomical sites, corresponding to 141,085,065 unique tissue tiles. The held-out test set comprised an additional 1,109 WSIs from 905 patients and 15,179,191 tissue tiles. In total, the dataset contained 11,593 WSIs from 9,546 patients and 156,264,256 unique tissue tiles.

Slides were processed at 20× magnification and partitioned into non-overlapping 256 × 256-pixel tiles. Tiles with insufficient tissue content, determined from the proportion of non-background pixels, were excluded before feature extraction. No stain or color normalization was applied to the image tiles, and no additional normalization was applied to the extracted foundation-model features before SAE training.2 All slides from the same patient were assigned to the same partition to prevent patient-level information leakage.

Let tm,it_{m,i} denote tissue ii from slide mm, where i=1,…,Nmi = 1, \ldots, N_m, with NmN_m as the number of tiles for slide mm. Each retained tile was encoded using the frozen UNI2-h pathology foundation model fUNI2f_{\mathrm{UNI2}} Chen et al. [ [11] to obtain a D=1,536D = 1,536-dimensional feature representation,

xm,i=fUNI2(tm,i),xm,i∈RD.(1)x_{m,i} = f_{\mathrm{UNI2}}(t_{m,i}), \qquad x_{m,i} \in\mathbb{R}^{D}. \tag*{(1)}

For slide mm, the resulting tile features were stacked as

Xm=[xm,1,…,xm,Nm]T∈RNm×D.X_m = [x_{m,1}, \ldots, x_{m,N_m}]^{\mathsf{T}} \in\mathbb{R}^{N_m \times D}.

The UNI2-h encoder remained frozen throughout SAE training, downstream model training, and counterfactual evaluation. SAE parameters and concept prototypes were learned exclusively from the SAE training partition; the held-out partition was not used for SAE training, concept-prototype construction, or hyperparameter tuning. Task-specific downstream models were similarly developed using patient-disjoint training and test partitions.

End-to-end counterfactual generation

The end-to-end SCALE workflow follows the seven stages summarized in fig:1c. Steps 1–3 define what morphology should be changed for a specified alternative prediction. In step 1, a sparse autoencoder (SAE) decomposes pathology foundation-model representations into sparse latent components capturing recurrent morphological patterns. In step 2, these components are then grounded in image space using their top-activating tissue tiles, forming a bank of morphological concepts reusable across cancer types (more details in Sec. 4.3). In step 3, for a specified source-to-target counterfactual direction, concept-class associations are used to identify the concept most strongly associated with the target label (e.g., HPV+); this concept, together with its latent prototype, serves as the morphological target for counterfactual intervention (more details in § 4.4).

Once the target morphology has been identified, step 4 translates this concept into a feature-space intervention. The SAE representation of each selected tile is steered toward the latent prototype of the target concept and then decoded back into the pathology foundation-model feature space. This produces a modified feature representation that preserves the original tile representation while shifting it toward the selected morphological alternative, e.g., shifting from HPV+ to HPV− (more details in § 4.5).

Step 5 performs this feature-space intervention in image space. Specifically, the steered feature from step 4 is provided as conditioning (input) to a frozen feature-conditioned diffusion model, which generates the corresponding morphological counterfactual explanation image. The locations selected for intervention are prioritized using attention from the same MIL model whose prediction is being explained (Figure 1b), thereby determining where the target-associated morphology is introduced. Surrounding tissue is retained as spatial context during generation so that the induced morphological change remains localized to the intended region (more details in § 4.5).

When the requested counterfactual extends beyond the field of view of a single image generation window, step 6 propagates these local interventions across larger tissue regions, potentially to the entire slide. SCALE applies overlapping generation windows progressively across the selected locations, carrying the continuously updated tissue image forward so that previously generated morphology provides context for subsequent edits. This progressive procedure determines how localized morphological changes are extended across larger spatial extents and toward whole-slide scale while maintaining continuity between neighboring regions (more details in § 4.6).

Finally, step 7 closes the counterfactual explanation loop. The generated counterfactual tissue is re-encoded using the same frozen tile encoder and returned to the same MIL model used for the original prediction. The resulting change in target-class probability measures how the model responds to the introduced morphological alternative (i.e., the counterfactual explanation). Thus, SCALE connects the morphology selected for intervention, the tissue location at which it is introduced, and its spatial realization in image space to the resulting change in model prediction.

Sparse autoencoder and concept prototypes

To obtain a sparse representation of pathology features, we trained a sparse autoencoder (SAE) Cunningham et al. [2023] on tile-level UNI2-h representations from the SAE training partition. For notational simplicity, we write xix_i for the feature of tile ii, suppressing its slide index mm. Given an input feature xi∈RDx_i \in\mathbb{R}^{D}, the SAE encoder produced an overcomplete latent representation

zi=ReLU⁡(Wencxi+benc),zi∈RL,(2)z_i = \operatorname{ReLU}(W_{\mathrm{enc}}x_i + b_{\mathrm{enc}}), \qquad z_i \in\mathbb{R}^{L}, \tag*{(2)}

where L=12,288>D=1,536L = 12,288 > D = 1,536. The ReLU activation constrained SAE activations to be non-negative. The corresponding decoder reconstructed the pathology feature as

x^i=Wdecz^i+bdec.(3)\hat{x}_i = W_{\mathrm{dec}}\hat{z}_i + b_{\mathrm{dec}}. \tag*{(3)}

The SAE was optimized using a reconstruction objective with an ℓ1\ell_1 sparsity penalty,

LSAE=1B∑i=1B∥xi−x^i∥22+λSAE1B∑i=1B∥zi∥1,(4)\mathcal{L}_{\mathrm{SAE}} = \frac{1}{B}\sum_{i=1}^{B}\lVert x_i - \hat{x}_i\rVert_2^2 + \lambda_{\mathrm{SAE}}\frac{1}{B}\sum_{i=1}^{B}\lVert z_i\rVert_1, \tag*{(4)}

where BB denotes the number of tile features in a training batch and λSAE\lambda_{\mathrm{SAE}} controls the strength of sparsity regularization. Because ziz_i is non-negative, the ℓ1\ell_1 term penalizes total activation magnitude and encourages each tile to be represented by a relatively small subset of active SAE dimensions.

Each SAE latent dimension was treated as a candidate morphological concept. For concept jj, training tiles were ranked according to their activation zi,jz_{i,j} in the corresponding latent dimension. We used the Kproto=10K_{\mathrm{proto}} = 10 highest-activating tiles to represent each concept, denoting the tile set as

Tj=TopK⁡({zi,j}i=1Ntr;Kproto),(5)\mathcal{T}_j = \operatorname{TopK}\left(\left\{z_{i,j}\right\}_{i=1}^{N_{\mathrm{tr}}}; K_{\mathrm{proto}}\right), \tag*{(5)}

where NtrN_{\mathrm{tr}} is the number of tiles in the SAE training partition and TopK⁡(⋅;K)\operatorname{TopK}(\cdot; K) returns the indices of the KK largest values in the set. The corresponding image tiles were retained as visual examples of the morphology associated with concept jj.

For counterfactual steering, we additionally constructed a latent prototype for each concept jj by averaging the complete SAE codes of these top-activating tiles,

pj=1∣Tj∣∑i∈Tjzi,pj∈RL.(6)p_j = \frac{1}{\lvert\mathcal{T}_j \rvert}\sum_{i\in\mathcal{T}_j} z_i,\qquad p_j \in\mathbb{R}^{L}. \tag*{(6)}

Thus, the scalar activation zi,jz_{i,j} was used to identify tiles strongly expressing concept jj, whereas pjp_j retained the joint latent configuration characteristic of those representative tiles. We used this complete latent prototype pjp_j as the steering target because the morphology represented by a concept may depend on the configuration of multiple SAE dimensions rather than on a single latent activation alone. This choice was further evaluated against direct single-neuron steering in an ablation experiment (Supplementary Fig. 4). Concept prototypes were constructed exclusively from the SAE training partition and remained fixed during downstream counterfactual experiments.

Attention-based MIL and concept-class association

Task-specific attention-based multiple-instance learning (MIL) models were trained for breast, colorectal, kidney, lung, head-and-neck and prostate pathology using patient-disjoint training and test partitions. The evaluated tasks comprised BRCA, COAD, KIRC and LUAD normal-versus-tumor classification; HNSCC HPV-status classification; and a three-class PRAD growth-pattern task comprising GP1–3, GP4 and GP5. For PRAD, GP1–3 denotes an experimental grouping of discrete, well-formed glands based on the historical Gleason patterns 1–3; patterns 1 and 2 are no longer routinely assigned in contemporary clinical grading Epstein et al. [ [19].

For slide mm with NmN_m tiles, the set of tile features {xm,i}i=1Nm\{x_{m,i}\}_{i=1}^{N_m}, xm,i∈Rdx_{m,i} \in\mathbb{R}^{d}, was treated as a bag of instances. The corresponding MIL model assigned normalized attention weights am,ia_{m,i} to individual tiles, where

am,i≥0,∑i=1Nmam,i=1,a_{m,i} \ge0,\qquad\sum_{i=1}^{N_m} a_{m,i}=1,

and aggregated the tile representations into a slide-level representation by attention-weighted averaging,

hm=∑i=1Nmam,ixm,i,hm∈Rd,h_m=\sum_{i=1}^{N_m} a_{m,i}x_{m,i},\qquad h_m\in\mathbb{R}^{d},

which was passed to a classification head to obtain the slide-level prediction. Attention weights therefore provided a model-derived measure of the relative contribution of each tissue tile to the slide-level prediction.

To identify morphological concepts associated with each downstream class, SAE activations were combined with attention weights from the corresponding trained MIL model. Let zm,i,jz_{m,i,j} denote the activation of concept jj for tile ii from slide mm. We defined the attention-weighted slide-level activation of concept jj as

qm,j=∑i=1Nmam,izm,i,j.(7)q_{m,j}=\sum_{i=1}^{N_m}a_{m,i}z_{m,i,j}. \tag*{(7)}

Because ∑i=1Nmam,i=1\sum_{i=1}^{N_m}a_{m,i}=1, qm,jq_{m,j} is a weighted mean over tiles and its scale does not depend on the number of tiles NmN_m, so slides with larger size do not receive larger scores.

For class cc, let Mctrain\mathcal{M}^{\mathrm{train}}_c denote the set of training slides with label cc. The mean attention-weighted activation of concept jj within class cc was

μj(c)=1∣Mctrain∣∑m∈Mctrainqm,j.(8)\mu_j^{(c)}=\frac{1}{\lvert\mathcal{M}^{\mathrm{train}}_c\rvert}\sum_{m\in\mathcal{M}^{\mathrm{train}}_c}q_{m,j}. \tag*{(8)}

Averaging within each class in Eq. (8) normalizes by the class size ∣Mctrain∣\lvert\mathcal{M}^{\mathrm{train}}_c\rvert, so classes with more training slides do not disproportionately influence concept-association estimates. Together, Eqs. (7) and (8) therefore reduce bias from unequal slide size and unequal class size.

For a counterfactual intervention from source class csrcc_{\mathrm{src}} to target class ctgtc_{\mathrm{tgt}}, we defined the concept–class association contrast as

Γj(ctgt←csrc)=μj(ctgt)−μj(csrc).(9)\Gamma_j^{(c_{\mathrm{tgt}}\gets c_{\mathrm{src}})}=\mu_j^{(c_{\mathrm{tgt}})}-\mu_j^{(c_{\mathrm{src}})}. \tag*{(9)}

Positive values therefore indicate concepts with stronger attention-weighted expression in the target class than in the source class. The target-associated concept used for steering was then selected as

j∗=arg⁡max⁡jΓj(ctgt←csrc).(10)j^* = \arg\max_j \Gamma^{(c_{\mathrm{tgt}}\leftarrow c_{\mathrm{src}})}_j. \tag*{(10)}

Note that all concept-class associations above were computed using training slides only, and the selected concept j∗j^* for each source-to-target intervention direction was fixed before evaluation on held-out slides. Thus, the pan-cancer SAE concept bank remained unchanged across a wide range of downstream tasks, while the morphological concept selected for counterfactual intervention was determined separately for each task and intervention direction.

Attention-guided concept steering

For counterfactual intervention, non-background tissue tiles within the evaluated region were ranked according to attention weights from the corresponding MIL model. Let Vm\mathcal{V}_m denote the set of non-background tissue tiles in slide mm, with Nmtissue=∣Vm∣N_m^{\mathrm{tissue}} = |\mathcal{V}_m|, and let (i1,…,iNmtissue)(i_1,\ldots,i_{N_m^{\mathrm{tissue}}}) denote these tiles ordered such that

am,i1≥am,i2≥⋯≥am,iNmtissue.(11)a_{m,i_1} \ge a_{m,i_2} \ge\cdots\ge a_{m,i_{N_m^{\mathrm{tissue}}}}. \tag*{(11)}

For an editing budget of kk tiles, the attention-guided edit set was

Iedit,m(k)={i1,…,ik}.(12)\mathcal{I}_{\mathrm{edit},m}^{(k)} = \{i_1,\ldots,i_k\}. \tag*{(12)}

For the random-selection baseline, the same number of tiles was sampled uniformly at random from Vm\mathcal{V}_m. The target concept, steering parameters, diffusion generator and editing budget were otherwise held fixed, isolating the effect of spatial selection.

Let j∗j^* denote the selected target-associated concept and pj∗p_{j^*} its latent prototype. For notational simplicity, we omit the slide index mm below. For each selected tile i∈Iediti \in\mathcal{I}_{\mathrm{edit}}, the original SAE code ziz_i was interpolated toward the target prototype,

zi′=zi+s(pj∗−zi)=(1−s)zi+spj∗,0≤s≤1,(13)z_i' = z_i + s\left(p_{j^*} - z_i\right) = (1-s)z_i + sp_{j^*}, \qquad0 \le s \le1, \tag*{(13)}

where ss denotes the SAE steering strength. Setting s=0s = 0 leaves the original latent representation unchanged, whereas s=1s = 1 replaces it with the target prototype.

The steered latent representation was decoded back into pathology-feature space and interpolated with the original UNI2-h feature,

xi′=(1−β)xi+β(Wdeczi′+bdec),0≤β≤1,(14)x_i' = (1-\beta)x_i + \beta\left(W_{\mathrm{dec}}z_i' + b_{\mathrm{dec}}\right), \qquad0 \le\beta\le1, \tag*{(14)}

where β\beta controls the contribution of the decoded steered feature to the final conditioning representation. Unless otherwise specified, counterfactual experiments used steering strength s=0.9s = 0.9 and feature-mixing coefficient β=0.1\beta= 0.1. Steering strength was varied only in the intervention-strength analysis. Tiles outside the edit set retained their original features,

xi′=xi,i∉Iedit.(15)x_i' = x_i, \qquad i \notin\mathcal{I}_{\mathrm{edit}}. \tag*{(15)}

The resulting features were returned to their original spatial locations to form the edited conditioning grid FeditF_{\mathrm{edit}}, with each grid cell corresponding to one spatially aligned 256×256256 \times256-pixel pathology tile. In other words, each returned UNI2-h feature vector xi′x_i' will be fed into a pathology diffusion model Yellapragada et al. [2025] to generate one 256×256256 \times256-pixel pathology tile, laying the foundation for Sec. 4.6 below.

Progressive diffusion-based counterfactual generation

Counterfactual images were generated using a frozen feature-conditioned pathology diffusion model Yellapragada et al. [2025] that operates on a spatial 4×44 \times4 grid of conditioning features, with each cell corresponding to a 256×256256 \times256 tile. To extend generation beyond this native field of view, we developed a progressive overlapping-window procedure that coordinates successive local interventions through shared spatial context.

Let I(0)I^{(0)} denote the original source region. Each conditioning-grid cell corresponded to one spatially aligned 256×256256 \times256-pixel pathology tile. Generation proceeded through overlapping 4×44 \times4-tile local windows (1024×10241024 \times1024 pixels), each containing a fully edited 2×22 \times2 center surrounded by a blended peripheral region (Figure 6a). By traversing these overlapping windows across the tissue, editing was extended over the full 8×88 \times8 global conditioning grid that constitutes each 2048×20482048 \times2048-pixel region used for large-region evaluation (Figure 6e).

For each active window, SAE-steered conditioning features xi′x_i^{\prime} were applied only to selected tiles (Figure 6d) when they entered the 2×22 \times2 editable center (Figure 6a). Unselected cells within the editable center and all cells in the peripheral region retained their original features xix_i. Concept-guided intervention was therefore confined to the intended spatial locations, while surrounding conditioning features provided local morphological context.

At progressive step rr, the diffusion generator operated on the corresponding region of the current composite image I(r−1)I^{(r-1)} (e.g., a large 2048×20482048 \times2048-pixel region or the whole slide). Generated pixels within the central editable region were incorporated fully into the composite image. Within the peripheral region, generated output was blended with the existing composite using 10% newly generated content and 90% existing content, while pixels outside the active window remained unchanged. The resulting image I(r)I^{(r)} was then carried forward to the next overlapping window.

Because each subsequent window operated on the progressively updated image rather than the original source region, previously generated tissue provided spatial context for later interventions. Windows were traversed until all selected tiles had entered a central editable region, allowing localized counterfactual changes to be propagated across larger tissue extents while maintaining continuity with neighboring tissue.

To evaluate extension beyond the standardized large 2048×20482048 \times2048-pixel regions, we applied the same progressive procedure to complete COAD WSIs. Attention-prioritized tissue locations were edited through successive overlapping local windows on a continuously updated slide image. Each generation step retained the same bounded 4×44 \times4 conditioning window used for large-region experiments. Counterfactual morphology was then progressively propagated across spatially distributed tissue regions while preserving the surrounding slide context.

For the independent-window baseline, we used the same diffusion generator, conditioning features, window geometry, traversal sequence and spatial composition rule. The only difference was that each local window was generated from the corresponding region of the original source image I(0)I^{(0)}, rather than from the progressively updated composite I(r−1)I^{(r-1)}. Previously generated tissue therefore did not provide context for subsequent windows.

Counterfactual evaluation

We evaluated SCALE at two spatial scales: whole-slide counterfactual editing as the primary evaluation and fixed-region counterfactual editing as a complementary analysis. Both settings used held-out samples from the designated source class of each intervention direction and measured how progressively increasing the spatial extent of counterfactual intervention affected predictions from the corresponding task-specific multiple-instance learning (MIL) model.

Whole-slide counterfactual evaluation. The primary evaluation included 134 distinct whole-slide images (WSIs) from 101 TCGA cases across nine counterfactual task directions. For each WSI, all non-background tissue tiles were considered eligible for intervention. Attention weights from the same task-specific MIL model whose prediction was being explained were used to rank eligible tissue tiles in descending order, prioritizing prediction-relevant regions for counterfactual editing.

To evaluate the effect of spatial intervention extent across WSIs containing different numbers of tissue tiles, editing budgets were expressed as percentages of the total eligible tissue tiles rather than fixed tile counts. Let Vm\mathcal{V}_m denote the set of eligible tissue tiles in slide mm, with ∣Vm∣|\mathcal{V}_m| representing the total number of eligible tiles. For an editing extent of p%p\%, the corresponding editing budget was defined as

km(p)=⌈p100∣Vm∣⌉,(16)k_m(p) = \left\lceil\frac{p}{100} |\mathcal{V}_m| \right\rceil, \tag*{(16)}

with km(0)=0k_m(0) = 0. The cumulative edit set Tedit,m(p)\mathcal{T}_{\mathrm{edit},m}^{(p)} contained the km(p)k_m(p) highest-ranked eligible tissue tiles. Edit sets were nested, such that tiles selected at smaller editing percentages remained included at larger intervention extents.

Counterfactual images were generated at the selected spatial locations using the progressive overlapping-window editing procedure, allowing concept-guided morphological interventions to extend across the WSI while preserving surrounding tissue context. At each editing extent, the generated counterfactual tissue tiles were re-encoded using the frozen UNI2-h encoder, and their features replaced the corresponding original tile features in the whole-slide feature bag. The resulting feature bag was then evaluated using the original task-specific MIL model to measure the target-class probability after intervention.

Region-level counterfactual evaluation. As a complementary evaluation, we examined counterfactual editing within fixed 2048×20482048 \times2048-pixel tissue regions. Candidate regions from multiple held-out source-class slides were first evaluated using the corresponding task-specific MIL model. Only regions for which the pre-intervention prediction matched the source slide label were retained. From this eligible pool, 30 regions were randomly sampled per slide for counterfactual evaluation.

Within each selected region, only non-background tissue tiles were eligible for intervention. Attention weights from the same MIL model were used to rank these tiles and determine their editing order.

To evaluate increasing intervention extent within the fixed regions, tiles were cumulatively edited using

k∈{1,8,16,32,48},(17)k \in\{1, 8, 16, 32, 48\}, \tag*{(17)}

where kk denotes the absolute number of edited tiles. The edit sets were nested, with tiles selected at smaller editing budgets retained at larger budgets. Let Vm,r\mathcal{V}_{m,r} denote the eligible tissue tiles in region rr of slide mm, and let Iedit,m,r(k)⊆Vm,r\mathcal{I}^{(k)}_{\mathrm{edit},m,r} \subseteq\mathcal{V}_{m,r} denote the cumulative set of the kk highest-ranked tissue tiles. The corresponding generated counterfactual region is denoted by Im,r,kcfI^{\mathrm{cf}}_{m,r,k}.

Whole-slide prediction assessment. For both evaluation settings, generated counterfactual tiles were re-encoded using the same frozen UNI2-h encoder used for the original WSI. Let

Xm=[xm,1;…;xm,Nm]∈RNm×d(18)X_m = [x_{m,1}; \ldots; x_{m,N_m}] \in\mathbb{R}^{N_m \times d} \tag*{(18)}

denote the original whole-slide feature bag of slide mm, containing NmN_m tile features as introduced in § 4.4.

For the whole-slide evaluation, let Xmcf,(p)X_m^{\mathrm{cf},(p)} denote the counterfactual feature bag obtained after editing p%p\% of the eligible tissue tiles. For the region-level evaluation, let Xm,rcf,(k)X_{m,r}^{\mathrm{cf},(k)} denote the whole-slide feature bag after replacing features corresponding to the kk edited tiles within region rr. In both settings, features outside the respective edit sets remained unchanged.

The original task-specific MIL model was then applied to the resulting counterfactual feature bag. For target class ytgty_{\mathrm{tgt}}, the whole-slide target-class probability at editing extent pp was defined as

πtgt,m(p)=P(ytgt∣Xmcf,(p)).(19)\pi_{\mathrm{tgt},m}(p) = P\left(y_{\mathrm{tgt}} \mid X_m^{\mathrm{cf},(p)}\right). \tag*{(19)}

Analogously, region-level interventions were evaluated by applying the same MIL model to Xm,rcf,(k)X_{m,r}^{\mathrm{cf},(k)}. This procedure quantified the extent to which counterfactual morphological interventions shifted the predictions of the original model being explained, without retraining or modifying its parameters.

The target-directed probability change relative to the original WSI was defined as

Δπtgt,m(p)=πtgt,m(p)−πtgt,m(0).(20)\Delta\pi_{\mathrm{tgt},m}(p) = \pi_{\mathrm{tgt},m}(p) - \pi_{\mathrm{tgt},m}(0). \tag*{(20)}

Positive values indicate that the counterfactual intervention increased the model’s predicted probability of the requested target class. The same evaluation principle was applied to region-level interventions using the corresponding counterfactual whole-slide feature bags.

The benchmark comprised nine source-to-target intervention directions: BRCA normal-to-tumor, COAD normal-to-tumor, KIRC normal-to-tumor, LUAD normal-to-tumor, HNSCC HPV-positive-to-HPV-negative, HNSCC HPV-negative-to-HPV-positive, PRAD GP1–3-to-GP4, PRAD GP4-to-GP5 and PRAD GP1–3-to-GP5. For each direction, the corresponding MIL model, target-associated concept and steering parameters were fixed before evaluation on held-out data.

To separately examine the effects of local steering strength and spatial editing extent, we performed two additional controlled analyses on representative tissue regions. For the steering-strength analysis, the spatial intervention region was held fixed while the SAE steering strength was varied from 0% to 100%. For the editing-extent analysis, steering parameters were held fixed while the fraction of non-background tissue tiles selected for intervention was increased from 0% to 100%. Target-class probability and mean activation of the selected target concept were recorded after each intervention.

Visual perturbation and boundary analysis

Visual perturbation was measured between spatially aligned source and counterfactual regions. Let Ω\Omega denote the evaluated image domain. The mean absolute channel-wise RGB difference was defined as

ERGB(k)=13∣Ω∣∑u∈Ω∥Ikcf(u)−I(0)(u)∥1(21)\mathcal{E}_{\mathrm{RGB}}(k)=\frac{1}{3|\Omega|}\sum_{u\in\Omega}\left\lVert I_k^{\mathrm{cf}}(u)-I^{(0)}(u)\right\rVert_1 \tag*{(21)}

where I(0)(u)∈R3I^{(0)}(u)\in\mathbb{R}^{3} denotes the RGB intensity vector of the source region at pixel uu, and Ikcf(u)∈R3I_k^{\mathrm{cf}}(u)\in\mathbb{R}^{3} denotes the RGB intensity vector of the counterfactual region at the same pixel location after editing kk tiles. The factor 1/31/3 averages the absolute difference over the three color channels. RGB intensities were represented on the [0,255][0,255] scale, so ERGB\mathcal{E}_{\mathrm{RGB}} is reported in RGB intensity units on the same scale. Lower values indicate less pixel-level deviation from the source image.

To quantify discontinuities introduced where neighboring generation windows met, let B\mathcal{B} denote the set of adjacent pixel pairs (u,v)(u,v) lying on opposite sides of the evaluated window boundaries. For image JJ, boundary discontinuity was defined as

BD(J)=13∣B∣∑(u,v)∈B∥J(u)−J(v)∥1(22)\mathrm{BD}(J)=\frac{1}{3|\mathcal{B}|}\sum_{(u,v)\in\mathcal{B}}\left\lVert J(u)-J(v)\right\rVert_1 \tag*{(22)}

where J(u)∈R3J(u)\in\mathbb{R}^{3} and J(v)∈R3J(v)\in\mathbb{R}^{3} denote the RGB intensity vectors of JJ at the two pixels of a boundary pair, so that BD(J)\mathrm{BD}(J) is the mean absolute channel-wise intensity difference across window boundaries. Excess boundary discontinuity relative to the spatially aligned source image was

Eborder(k)=BD(Ikcf)−BD(I(0)).(23)\mathcal{E}_{\mathrm{border}}(k)=\mathrm{BD}\left(I_k^{\mathrm{cf}}\right)-\mathrm{BD}\left(I^{(0)}\right). \tag*{(23)}

The same boundary locations were used when evaluating progressive and naive independent-window generation. A value of zero indicates boundary discontinuity equal to that already present in the source image, whereas positive values indicate additional discontinuity introduced by generation. Lower ERGB\mathcal{E}_{\mathrm{RGB}} and Eborder\mathcal{E}_{\mathrm{border}} therefore indicate less visual perturbation and lower excess boundary discontinuity, respectively.

For each editing budget, visual metrics were calculated separately for each task direction and then averaged across the nine intervention directions. Shaded uncertainty bands denote the standard error of the mean across task directions (n=9n=9), with task direction serving as the unit of cross-task summary.

References

  1. [1]Nicolas Coudray, Paolo Santiago Ocampo, Theodore Sakellaropoulos, Nayneet Narula, Matija Snuderl, David Fenyö, Andre L. Moreira, Narges Razavian, and Aristotelis Tsirigos. Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nature Medicine, 24:1559–1567, 2018. doi:10.1038/s41591-018-0177-5.
  2. [2]Gabriele Campanella, Matthew G. Hanna, Luke Geneslaw, Allen Miraflor, Vitor Werneck Krauss Silva, Klaus J. Busam, Edi Brogi, Victor E. Reuter, David S. Klimstra, and Thomas J. Fuchs. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nature Medicine, 25:1301–1309, 2019. doi:10.1038/s41591-019-0508-1.
  3. [3]Jakob Nikolas Kather, Alexander T. Pearson, Niels Halama, Dirk Jäger, Jeremias Krause, Sven H. Loosen, Alexander Marx, Peter Boor, Frank Tacke, Ulf Peter Neumann, Heike I. Grabsch, Takaki Yoshikawa, Hermann Brenner, Jenny Chang-Claude, Michael Hoffmeister, Christian Trautwein, and Tom Luedde. Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer. Nature Medicine, 25:1054–1056, 2019. doi:10.1038/s41591-019-0462-y.
  4. [4]Ming Y. Lu, Drew F. K. Williamson, Tiffany Y. Chen, Richard J. Chen, Matteo Barbieri, and Faisal Mahmood. Data-efficient and weakly supervised computational pathology on whole-slide images. Nature Biomedical Engineering, 5:555–570, 2021. doi:10.1038/s41551-020-00682-w.
  5. [5]Mina Jamshidi Idaji, Julius Hense, Tom Neuhäuser, Augustin Krause, Yanqing Luo, Oliver Eberle, Thomas Schnake, Laure Ciernik, Farnoush Rezaei Jafari, Reza Vahidimajd, Jonas Dippel, Christoph Walz, Frederick Klauschen, Andreas Mock, and Klaus-Robert Müller. Beyond attention heatmaps: How to get better explanations for multiple instance learning models in histopathology. Medical Image Analysis, 113:104148, 2026. doi:10.1016/j.media.2026.104148.DOI
  6. [6]Weiyi Qin, Yingci Liu-Swetz, Shiwei Tan, and Hao Wang. Clear-hpv: Interpretable concept discovery for human papillomavirus-associated morphology in whole-slide histology. Patterns, page 101588, 2026. ISSN 2666-3899. doi:10.1016/j.patter.2026.101588.
  7. [8]Laura Zigutyte, Tim Lenz, Tianyu Han, Nic G. Reitsam, Sebastian Foersch, Katherine J. Hewitt, Moritz Jesinghaus, Zunamys I. Carrero, Michaela Unger, Asier Rabasco Meneghetti, Georg Lurje, Isabella Lurje, Sophia Herda, Justus Pein, Deniz Uluk, Carolin V. Schneider, Alexander T. Pearson, Daniel Truhn, and Jakob Nikolas Kather. Counterfactual diffusion models provide interpretable explanations of artificial intelligence models in pathology. Cancer Research, 86(17):4414–4433, 09 2026. ISSN 0008-5472. doi:10.1158/0008-5472.CAN-25-5645. URL https://doi.org/10.1158/0008-5472.CAN-25-5645.
  8. [9]Jakub R. Kaczmarzyk, Chanwoo Kim, Soham Gadgil, Deepika Savant, Zhen Zhao, Joel H. Saltz, Su-In Lee, and Peter K. Koo. Explainable AI for computational pathology identifies model limitations and tissue biomarkers, 2024.arxiv.org/abs/2409.03080
  9. [10]Brandon Muñoz, Raquel Pezoa, and Helen Gutierrez. Histoxplaining in style: Counterfactual explanations for histopathology images using StyleGAN2-ADA. Information Systems Frontiers, 2026. doi:10.1007/s10796-026-10709-3.DOI
  10. [11]Rishard J. Chen et al. Towards a general-purpose foundation model for computational pathology. Nature Medicine, 30:850–862, 2024. doi:10.1038/s41591-024-02857-3.DOI
  11. [12]Katherine A. Hoadley et al. Cell-of-origin patterns dominate the molecular classification of 10,000 tumors from 33 types of cancer. Cell, 173(2):291–304, 2018. doi:10.1016/j.cell.2018.03.022.DOI
  12. [13]Matthew Fleming, Sreelakshmi Ravula, Sergei F. Tatischev, and Hanlin L. Wang. Colorectal carcinoma: Pathologic aspects. Journal of Gastrointestinal Oncology, 3(3):153–173, 2012. doi:10.3978/j.issn.2078-6891.2012.030.DOI
  13. [14]Ruqqayya Awan, Korsuk Sirinukunwattana, David Epstein, Samuel Jefferyes, Quyis Qidwai, Zia Aftab, Imaad Mujeeb, David Snead, and Nasir Rajpoot. Glandular morphometrics for objective grading of colorectal adenocarcinoma histology images. Scientific Reports, 7:16852, 2017. doi:10.1038/s41598-017-16516-w.DOI
  14. [15]Behtash G. Nezami and Gregory T. MacLennan. Clear cell renal cell carcinoma: A comprehensive review of its histopathology, genetics, and differential diagnosis. International Journal of Surgical Pathology, 33(2):265–280, 2025. doi:10.1177/10668969241256111.DOI
  15. [16]Maura L. Gillison, William M. Koch, Ralph B. Capone, Michael Spafford, William H. Westra, Ling Wu, Mark L. Zahurak, Robert W. Daniel, Michael Viglione, Daniel E. Symer, Keerti V. Shah, and David Sidransky. Evidence for a causal association between human papillomavirus and a subset of head and neck cancers. Journal of the National Cancer Institute, 92(9):709–720, May 2000. doi:https://doi.org/10.1093/jnci/92.9.709.
  16. [17]Kevin K. Ang, Jay Harris, Robert Wheeler, Randal Weber, David I. Rosenthal, P. F. Nguyen-Tân, William H. Westra, Christine H. Chung, R. Clay Jordan, Chaomei Lu, Hyejin Kim, Rebecca Axelrod, Carole C. Silverman, Kimberly P. Redmond, and Maura L. Gillison. Human papillomavirus and survival of patients with oropharyngeal cancer. The New England Journal of Medicine, 363(1):24–35, July 2010. doi:https://doi.org/10.1056/NEJMoa0912217.
  17. [18]Okekchukwu R. Linton, Michael G. Moore, Joseph S. Brigance, Chris A. Gordon, Don-John Summerlin, and Mark W. Mc-Donald. Prognostic significance of basaloid squamous cell carcinoma in head and neck cancer. JAMA Otolaryngology–Head & Neck Surgery, 139(12):1306–1311, 2013. doi:https://doi.org/10.1001/jamaoto.2013.5308.
  18. [19]Jonathan I. Epstein, Michael J. Zelefsky, Daniel D. Sjoberg, Joel B. Nelson, Lars Egevad, Cristina Magi-Galluzzi, Andrew J. Vickers, Anil V. Parwani, Victor E. Reuter, Samson W. Fine, James A. Eastham, Peter Wiklund, Misop Han, Chandana A. Reddy, Jay P. Ciezki, Tommy Nyberg, and Eric A. Klein. A contemporary prostate cancer grading system: A validated alternative to the gleason score. European Urology, 69(3):428–435, 2016a. doi:10.1016/j.eururo.2015.06.046.DOI
  19. [21]Nhat Minh Le, Ciyue Shen, Neel Patel, Chintan Shah, Darpan Sanghavi, Blake Martin, Alfred Eng, Daniel Shenker, Harshith Padigela, Raymond Biju, Syed Ashar Javed, Jennifer Hipp, John Abel, Harsha Pokkalla, Sean Grullon, and Dinkar Juyal. Learning biologically relevant features in a pathology foundation model using sparse autoencoders, 2024. URL https://arxiv.org/abs/2407.10785.
  20. [22]Xiaorui Su, Pengwei Hu, Dongxu Li, Bowei Zhao, Zhaomeng Niu, Thomas Herget, Philip S. Yu, and Lun Hu. Interpretable identification of cancer genes across biological networks via transformer-powered graph representation learning. Nature Biomedical Engineering, 9(3):371–389, 2025. doi:10.1038/s41551-024-01312-5.DOI
  21. [23]Weixin Wang, Yu Yang, Wei Deng, and Pan Xu. Inference-time alignment of diffusion models via trust-region iterative twisted sequential monte carlo, 2026. URL https://arxiv.org/abs/2605.25123.
  22. [24]Yu Yang, Yihong Guo, Anqi Liu, and Pan Xu. Cross-domain energy-guided diffusion generation for off-dynamics reinforcement learning, 2026. URL https://arxiv.org/abs/2605.24810.

Paper details

vlnc:W_mbs3pk9q
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents