WorksComputer Science
SheetAlign: Grounded Interaction and Stage-Isolated Learning for Spreadsheet Agents
Abstract
Spreadsheets are widely used for data analysis and supporting decision-making. Automating spreadsheet analysis while protecting sensitive data and controlling deployment costs motivates locally deployed open-weight large language models (LLMs). Meanwhile, complex spreadsheet tasks require both task reasoning and workbook understanding, while predefined operation interfaces impose additional tool-selection and invocation demands. We introduce SheetAlign, a spreadsheet manipulation framework with stage-isolated policy learning, organized around where to operate, what to change, and how to implement transformations. Task-guided inspection and API grounding anchor code generation in workbook observations and available operations, while stage-isolated learning optimizes program generation and review using edit-progress rewards. At inference, execution consistency selects programs through agreement among workbook states. On SpreadsheetBench, our grounded task interface improves Qwen3-4B's accuracy from 13.5% to 29.2% on average without training. With learning and execution consistency, SheetAlign achieves 32.8% accuracy on average on SpreadsheetBench and 30.1% on Domain-Spreadsheet, outperforming comparable baselines. The code will be made publicly available upon publication.
Introduction
Spreadsheets support data analysis and operational workflows across many domains. LLM agents allow users to perform these tasks through natural-language instructions without manually constructing formulas or scripts [15, 2]. Keeping sensitive workbooks local under limited computing budgets motivates compact, open-weight LLMs.
Challenge 1: Task-to-Edit Grounding. Real-world spreadsheet requests often include previous attempts, encountered problems, and examples of desired outputs [16]. Existing agents rely on predefined actions or program execution with iterative verification [15, 2] (Wang et al. 2026). Regardless of the interface, agents must identify relevant workbook regions, determine required edits, and map them to appropriate operations. Resolving these interdependent decisions within complex workbooks is particularly challenging for compact models with limited reasoning capabilities. As illustrated in Figure 1 (upper part), a program may execute successfully yet modify unintended cells. This motivates grounding code generation in focused workbook observations and relevant API references, guiding where and what to edit while preserving implementation flexibility.
Figure 1. Motivations for SheetAlign. Upper: Executable programs may modify unintended cells without task-to-edit grounding. Lower: Whole-task rewards overlook partially completed edits.
Challenge 2: Limited Training Feedback. Spreadsheet tasks often require multiple coordinated edits, and partially correct transformations may still fail the overall task. As shown in Figure 1 (lower part), outcome-based rewards cannot distinguish different degrees of partial completion, providing limited guidance on remaining edits [4]. Moreover, program generation and review involve distinct decisions: generation constructs the requested transformation through programs, while program review identifies remaining intent misalignment for generated programs without disrupting correct edits. These observations motivate our edit-progress rewards and stage-isolated learning, as detailed in § 3.
Challenge 3: Generation Instability. Due to the stochastic nature of LLMs, repeated generations for the same spreadsheet task may yield both correct and incorrect transformations (Wang et al., 2023). This variability makes individual predictions unreliable, motivating execution consistency to improve accuracy and reliability through agreement among resulting workbooks.
We introduce SheetAlign, a spreadsheet manipulation framework combining a grounded task interface, stage-isolated policy learning, and execution consistency. It organizes each task around where to operate, what to change, and how to implement the transformation. Task-guided inspection and API grounding guide code generation with focused workbook observations and relevant API references, while execution feedback supports diagnostic repair. We separately train a generation policy to construct transformations and a review policy to refine executable programs, both using edit-progress rewards. At inference, execution consistency selects a program through agreement among resulting workbooks, followed by a review pass to address intent misalignment.
Our contributions are summarized as follows: (1) Grounded task interface. We introduce an interface that connects task-specific edit requirements with focused workbook observations and relevant APIs to guide code generation. (2) Stage-isolated policy learning. We separately optimize generation and review policies using edit-progress rewards to recognize partial completion and address remaining intent misalignment. (3) Execution consistency and empirical validation. We leverage agreement among resulting workbooks to mitigate generation instability and demonstrate its effectiveness on real-world spreadsheet benchmarks.
Related Work
Spreadsheet Agents. Existing spreadsheet agents employ different approaches to spreadsheet manipulation. InstructExcel generates Office Scripts from natural-language instructions [18], while SheetCopilot plans over predefined atomic actions [15]. SheetAgent extends these capabilities through planning, information gathering, and program execution [2]. Other work emphasizes workbook understanding. SpreadsheetLLM develops compact representations of large and irregular spreadsheets [6], while SheetBrain integrates query-aware workbook understanding with Python execution and validation (Wang et al., 2026). Recent benchmarks further cover real-world manipulation, business workflows, financial modeling, and workbook artifact creation [16, 11] (Zhu et al. 2026; Uniyal et al. 2026). Our work instead emphasizes explicit task-to-edit grounding, identifying what to read, where to write, and which operations to perform before code generation.
Spreadsheet Action Interfaces. At the execution level, spreadsheet agents use predefined actions or executable programs. Predefined actions provide structured operations, while program-based interfaces support compositional transformations [15, 18, 2]. Spreadsheet-RL combines spreadsheet-native tools with sandboxed Python execution in a multi-turn environment [4]. Beyond spreadsheets, CodeAct demonstrates executable code as a general agent action interface (Wang et al., 2024). However, action interfaces alone cannot specify how task-specific edits should be implemented in the current workbook. To address this, SheetAlign grounds identified edits in focused workbook observations and relevant API references to guide program generation.
Learning for Spreadsheet Generation and Revision. Execution-based learning has been explored for program generation, including spreadsheet transformations. CodeRL uses functional correctness feedback [13], while Spreadsheet-RL trains spreadsheet agents using execution and workbook-level outcomes [4]. Such feedback provides limited guidance on which required workbook edits have been completed. For revision, Self-Debugging refines programs through execution feedback and explanations [1], while SCoRe trains language models for self-correction [10]. Building on these approaches, SheetAlign combines diagnostic repair for execution failures with a dedicated review policy for intent misalignment in executable programs. Stage-isolated learning further optimizes generation and review separately using edit-progress rewards.
Execution-Based Candidate Selection. Prior work has explored consistency across multiple candidates to improve reliability. Self-consistency selects answers through agreement across sampled reasoning paths (Wang et al., 2023), while other approaches compare execution outputs or train candidate selectors [14] (Sheng and Xu 2025; Wang et al. 2025) . Spreadsheet programs may modify cell values, formulas, and structure, making agreement over the resulting workbooks more informative than answer-level agreement. Accordingly, SheetAlign selects programs by comparing their resulting workbooks.
Method
Given a natural-language instruction , an input workbook , and task-provided target locations , our goal is to produce a transformed workbook satisfying the user’s requirements. The target locations specify where results are expected, such as a worksheet cell or range, while the requested changes may involve values, formulas, formatting, or workbook structure. We introduce SheetAlign (shown in Figure 2), a spreadsheet agent built around three components: a grounded task interface, stage-isolated policy learning, and execution consistency selection.
Figure 2. Overview of SheetAlign, integrating task grounding, stage-isolated learning, and execution consistency.
The model expresses each transformation as Python code using openpyxl, while a shared harness provides task-relevant workbook observations and API references, executes generated programs, and returns diagnostics. Within this harness, we study two separately trained policies: a generation policy that produces spreadsheet transformations and a program review policy that revisits generated programs. The policies are trained independently, allowing generation and review to be studied separately. At inference, we select among multiple complete candidates by agreement between their resulting workbook states.
Grounded Task Interface
Task-guided workbook inspection. Before generating code, the model summarizes the task requirements and relevant workbook information using a compact template:
| Where | target region to modify |
| From | workbook context to inspect |
| What | requested transformation |
| Write as | values, formulas, formatting, or structure |
| API | relevant openpyxl functionality |
The template guides workbook inspection and API retrieval for subsequent code generation. For example, if the instruction asks to compute values from two source columns and write it to a target column, the description identifies the source columns, the target region, and the required computation. The harness then provides focused workbook observations, such as nearby headers, cells, formulas, and merged regions. These observations ground code generation in the current workbook state. The full structured representation and observation procedure are given in Appendix § 6.§ 1.
API grounding. The same task summarization also identifies the APIs needed for the task. The harness retrieves a small set of relevant openpyxl API references and provides them to the model together with the focused workbook observations. The model thus receives both the workbook context needed to understand the task and the API guidance needed to implement the transformation, while retaining the flexibility of code generation.
Execution and diagnostic repair. The harness executes each generated program and checks for execution failures or invalid workbook outputs. When an issue is detected, a concise diagnostic describing the problem, together with the current program and relevant workbook context, is provided to the generation policy introduced later. The same policy revises the program, which is rerun from the original workbook, with up to two repair attempts by default. This failure-driven repair differs from the separately trained review policy, also introduced later, which targets intent misalignment in executable programs through a single review pass. Further details are provided in Appendix § 6.§ 2.
Stage-Isolated Policy Learning
We train two separate policies from the same backbone LLM. The generation policy produces spreadsheet programs, while the review policy examines an executable program and decides whether to keep or revise it. We first obtain a common starting policy and then train the two policies independently. Each policy is optimized for its own role.
Shared optimization. We optimize each trainable policy with group relative policy optimization (GRPO) (Shao et al., 2024) and use LoRA [8] for parameter-efficient updates. For each prompt, GRPO samples (default by 4) responses and compares their rewards within the group. If response receives reward , its group-relative advantage is
where and are the mean and standard deviation of rewards for the same prompt. A positive indicates that the response performs better than the group average.
Let denote the token-level probability ratio between the updated policy and the policy used to sample response . We define its clipped form as
where controls the allowed change in the policy. We then optimize
The clipped objective limits overly large policy updates while favoring responses with higher relative rewards. Full optimization details and training settings are provided in Appendix § 6.§ 3.
Common starting policy. We first train a common starting policy with a diagnostic reward based on task correctness , execution success , and output quality , with an additional penalty for failed programs. We summarize the reward as
where penalizes execution failures according to their type, and , , and are fixed coefficients. The implementation applies these terms differently depending on the program outcome. This policy provides the common starting point for the generation policy and review policy branches. Appendix § 6.§ 4 gives the exact piecewise reward and coefficients.
Generation policy training. The learns broad task correctness and program execution, while the generation policy training focuses on tasks whose solutions remain unstable. For each training task, we sample multiple programs and prioritize cases that contain both a correct program and an executable but incorrect one. These cases show that the can solve the task, but does not yet produce the intended result consistently. We also sample coverage tasks across operation categories such as formulas, formatting, lookup, and structural edits to maintain task diversity.
Edit-level reward. Whole-task correctness overlooks partially completed solutions. We evaluate progress over required target-cell edits. Let be the number of required edits, the number of edits made by the program, and the number of required edits completed correctly. We compute
and use their harmonic mean as the edit-level progress reward
Precision discourages unnecessary edits, while recall measures completed required edits. Let indicate task correctness. We then combine and as
where . During generation and review policy training, replaces the correctness term in the initial reward, while the execution and output quality terms remain unchanged.
Review policy training. Diagnostic repair handles clear implementation or execution failures, but an executable program may still perform the wrong transformation. We therefore train a separate review policy as a complementary correction stage. Given the user instruction, an executable program, and focused observations of the input and output workbooks covering task-relevant cells, headers, and formulas, the decides whether to keep the program or revise it.
Both the and are continued independently from the same common starting policy . The learns to produce better initial solutions, while the learns to correct existing executable solutions. Their training is separate, so updates to one do not affect the other.
For review training, we construct training examples from fixed executable programs. The either keeps the original program or generates a revised one. The resulting program is executed and evaluated with the same reward function used for generation training, since both branches optimize the correctness of the final workbook. Here, still serves as the correctness term.
Execution Consistency Selection
We further improve inference reliability by selecting among multiple candidate programs based on their resulting workbooks. We sample two candidates from the and two from the . Each candidate undergoes the same grounding, execution, and diagnostic repair procedure.
Workbook agreement. For each output workbook that the harness can open and inspect, we construct a deterministic fingerprint from worksheet structure, cell locations, stored values and formulas, and formatting information. Two candidates agree when their fingerprints match, even if their generated programs differ. We select the first candidate from the largest agreement group containing at least two candidates. If no candidates agree, we select the first candidate with a valid output, or the first candidate if none is valid. After consistency selection, the review policy retains or revises the selected program to address intent misalignment. Further details are provided in Appendix § 6.§ 5.
Experiments
We investigate the following research questions:
Q1. Overall Effectiveness. How does SheetAlign compare with existing agents on complex spreadsheet tasks? Q2. Grounded Interface. How does grounded execution affect reliability and correctness of spreadsheet transformations? Q3. Policy Learning. How do different training signals and policy roles influence spreadsheet manipulation performance? Q4. Multi-Candidate Consistency. How can candidate diversity and output agreement improve solution selection?
Experimental Setup
Datasets. Following Spreadsheet-RL [4], we evaluate on two benchmarks. SpreadsheetBench [16] contains 912 complex real-world spreadsheet manipulation tasks with multiple workbook instances per task. Domain-Spreadsheet [4] contains 1,660 tasks across finance, supply chain, human resources, sales, and real estate. For training, we sample from 5,925 real-world Excel tasks released by Spreadsheet-RL [4].
Baselines. We compare with following baselines:
Closed-source systems. We include GPT-4o [17], OpenAI o3 [7], ChatGPT Agent [4], Microsoft Copilot Agent Mode [9], and Gemini (Team, 2025a) in Google Sheets. For GPT-4o, we report results using LibreOffice and Microsoft Excel. For ChatGPT Agent, we include both the standard configuration and the version with additional .xlsx file access. All results are taken from published evaluations or the official SpreadsheetBench leaderboard. Open-source models and agents. We include Qwen3 models of different sizes (Team, 2025b). We also compare with TabClaw ([3]) and its reported baselines, including SheetAgent ([2]), SpreadsheetLLM ([6]), and ReAct (Yao et al., 2023), all evaluated using DeepSeek-V3.2 ([5]).
Same-backbone comparisons. We compare four Qwen3-4B-Thinking-2507 configurations reported by Spreadsheet-RL ([4]): the base model, the model with a spreadsheet-native harness, the model with comprehensive spreadsheet tools, and outcome-based RL trained model. Using the same backbone, we also evaluate our SheetAlign framework both with and without training.
Implementation Details. We use Qwen3-4B-Thinking-2507 as the backbone model and execute spreadsheet transformations in Python using openpyxl. We train the policies with GRPO and LoRA using verl (Sheng et al., 2025) and vLLM ([12]). For evaluation, we recalculate generated workbooks using Microsoft Excel on Windows and compare them with reference workbooks using the official Spreadsheet-RL comparator. Training hyperparameters and inference settings are detailed in Appendix § 7.§ 1.
Evaluation Metrics. We report Average Accuracy (same as the Pass@1 results on Spreadsheet-RL) on both datasets, averaging instance accuracy across tasks. For SpreadsheetBench, which typically provides three related instances per task, we additionally report Hard Accuracy, requiring all instances to be correct. We also report Execution Success Rate, measuring the proportion of workbook instances on which the generated program executes successfully. All reported accuracy for our models are averaged over three random seeds.
Main Results (Q1)
SpreadsheetBench. Table 1 compares SheetAlign with existing spreadsheet agents. Using Qwen3-4B-Thinking-2507 as its backbone, our grounded task interface improves average accuracy from 13.5% to 29.5% without training, a gain of 16.0 percentage points. Stage-isolated learning further increases average accuracy to 30.4% and hard accuracy to 25.2%. The resulting average accuracy exceeds the 23.4% reported by Spreadsheet-RL with the same backbone by 7.0 percentage points. Execution consistency further improves average accuracy from 30.4% to 32.8% and hard accuracy
| Method | Hard | Avg. (%) | Exec. |
| Closed-source systems | |||
| GPT-4o (LibreOffice) | 16.8 | ||
| GPT-4o (Excel) | 18.4 | ||
| OpenAI o3 | 23.3 | ||
| ChatGPT Agent | 35.3 | ||
| ChatGPT Agent (.xlsx) | 45.5 | ||
| Copilot Agent Mode | 57.7 | ||
| Gemini in Google Sheets | 70.5 | ||
| Open-source Qwen3 models | |||
| Qwen3-4B-Instruct-2507 | 9.3 | ||
| Qwen3-4B | 11.0 | ||
| Qwen3-8B | 15.9 | ||
| Qwen3-14B | 15.0 | ||
| Qwen3-32B | 17.6 | ||
| Method | Hard | Avg. | Exec. |
| Spreadsheet agents (DeepSeek-V3.2) | |||
| ReAct | 16.1 | 18.8 | |
| SheetAgent | 18.5 | 22.3 | |
| SpreadsheetLLM | 19.8 | 23.3 | |
| TabClaw | 24.5 | 28.8 | |
| Qwen3-4B-Thinking-2507 | |||
| Spreadsheet-RL | |||
| + Native harness | 15.6 | ||
| + Comprehensive tools | 19.3 | ||
| + Outcome-based RL | 23.4 | ||
| Base model | 11.4 | 13.5 | 69.1 |
| SheetAlign (ours) | |||
| + Grounded task interface | 23.9 | 29.2 | 97.1 |
| + Stage-isolated learning | 25.2 | 30.4 | 97.7 |
| + Execution consistency | 27.1 | 32.8 | 98.6 |
Table 1. Main results on SpreadsheetBench (%). Avg. (Average Accuracy) measures the proportion of correctly solved workbook instances. Hard (Hard Accuracy) requires all instances of a task to be correct. Exec. (Execution Success Rate) measures successful program executions across instances. Execution consistency uses candidates.
from 25.2% to 27.1%, gains of 2.4 and 1.9 percentage points, respectively. Across three random seeds, SheetAlign shows a standard deviation of 0.4 percentage points in average accuracy. Execution success also increases from 97.7% to 98.6%. Overall, the grounded task interface provides the largest accuracy gain, with further improvements from policy training and execution consistency.
Domain-Spreadsheet. Table 2 reports results across seven domains. Our grounded task interface (training-free) achieves 24.9% average accuracy, exceeding the 17.2% reported by Spreadsheet-RL. Stage-isolated training improves average accuracy to 26.2%. Execution consistency selection raises it to 30.1%, a gain of 3.9 percentage points over SheetAlign without consistency.
| Domain | Base | RL | Harness | Learn. | Consist. |
| Finance-B | 15.6 | 29.3 | 36.7 | 37.0 | 42.9 |
| Finance-I | 7.7 | 16.2 | 26.8 | 28.1 | 31.2 |
| Finance-A | 8.1 | 19.3 | 22.2 | 27.4 | 28.8 |
| Supply Chain | 1.1 | 5.0 | 10.6 | 14.4 | 16.0 |
| HR | 0.5 | 3.2 | 13.5 | 14.1 | 14.6 |
| Sales | 1.2 | 5.8 | 8.1 | 8.1 | 11.5 |
| Real Estate | 1.1 | 1.1 | 10.1 | 11.2 | 19.1 |
| Overall | 8.4 | 17.2 | 24.9 | 26.2 | 30.1 |
Table 2. Average accuracy (%) on Domain-Spreadsheet using Qwen3-4B-Thinking-2507 as backbone. Harness denotes our grounded task interface, Learn. adds stage-isolated training, and Consist. applies execution consistency with candidate programs.
The consistency setting achieves the highest accuracy among our methods across seven domains, with improvements in Finance-B from 37.0% to 42.9% and real estate from 11.2% to 19.1%. Its execution success rate reaches 99.5%, compared with 92.1% for the grounded task interface. Across three random seeds, its average accuracy has a standard deviation of 0.3 percentage points.
Grounded Interface Analysis (Q2)
We examine the contribution of each component by removing it from the grounded task interface while keeping other settings fixed.
Table 3 shows that task-guided inspection has the largest impact on correctness, with its removal reducing average accuracy by 6.7 percentage points while execution success remains nearly unchanged. Removing API grounding decreases average accuracy by 3.2 points. Diagnostic repair improves both correctness and execution reliability, contributing 4.8 points to average accuracy and 6.6 points to execution success.
| Configuration | Hard | Avg. | Exec. |
| Full interface | 23.9 | 29.2 | 97.1 |
| w/o task-guided inspection | 17.2 | 21.4 | 96.7 |
| w/o API grounding | 20.7 | 25.2 | 95.8 |
| w/o diagnostic repair | 20.2 | 24.4 | 90.5 |
| Configuration | Avg. | Exec. | |
| Full interface | 24.9 | 92.1 | |
| w/o task-guided inspection | 23.3 | 89.6 | |
| w/o API grounding | 24.0 | 91.8 | |
| w/o diagnostic repair | 20.5 | 82.3 |
Table 3. Component ablations of the grounded task interface on SpreadsheetBench and Domain-Spreadsheet (%). Each variant removes one component from the full interface. Removing diagnostic repair retains diagnostic checks but disables LLM-based program correction.
On domain tasks, diagnostic repair has the largest impact, contributing 4.4 percentage points to average accuracy and 9.8 points to execution success. Task-guided inspection and API grounding contribute 1.6 and 0.9 points to average accuracy, respectively. Overall, task-guided inspection improves accuracy on SpreadsheetBench, while diagnostic repair enhances accuracy and execution reliability, particularly on Domain-Spreadsheet. Prompt templates are provided in Appendix § 7.§ 2.
Policy Learning Analysis (Q3)
We investigate how training signals and stage-isolated policy learning affect spreadsheet manipulation performance.
Training Signals. Table 4 compares our multi-signal reward in Eq. (4) with outcome-based reward inspired by Chi et al., [4] under the same harness. Our reward combines correctness, execution success, and program quality, providing richer supervision than outcome-based rewards. The common policy increases average accuracy from 29.1% to 29.7% on SpreadsheetBench and from 22.3% to 24.0% on Domain-Spreadsheet, suggesting benefits from richer training signals.
Stage-isolated Training. Starting from the common policy, generation and review are separately optimized using stage-isolated edit-progress rewards in Eq. (7). Both policies are optimized using stage-isolated edit-progress rewards. Generation training encourages completion of required workbook edits, while review training targets remaining intent misalignment in executable programs without disrupting correct edits. As shown in Table 4, generation specialization increases average accuracy from 29.3% to 30.0% on SpreadsheetBench and from 24.0% to 25.3% on Domain-Spreadsheet, while execution success improves from 96.4% to 97.0% and from 92.5% to 92.7%, respectively. Incorporating the review policy raises average accuracy to 30.4% and 26.2%, with execution success reaching 97.7% and 93.2%, respectively.
Training dynamics are presented in Appendix § 7.§ 4. Both policies show increasing rewards, with more pronounced fluctuations in generation, while review policy maintains higher execution success due to its focus on executable programs.
Candidate Consistency Analysis (Q4)
We evaluate candidate composition and selection strategies using four programs () sampled from either the training-free backbone or a mixture of common and generation policies.
Selection Strategies. The grounded task interface samples candidates from the original backbone without additional policy training. We compare execution consistency with two alternatives. First-valid selects the first executable candidate, while the learned selector uses Qwen3-4B with a trained LoRA adapter to rank candidates by predicted correctness. More details are in Appendix § 7.§ 5.
As shown in Table 5, multi-candidate consistency outperforms both first-valid and learned selection under the same candidate pool. On SpreadsheetBench, it achieves 27.1% hard accuracy and
| Candidate pool | Selection | Hard | Avg. | Exec. |
| Common + Generation | First-valid | 25.0 | 30.7 | 98.2 |
| Common + Generation | Learned | 25.8 | 31.4 | 98.4 |
| Grounded task interface | Consistency | 26.1 | 31.5 | 98.2 |
| Common policy | Consistency | 26.6 | 32.4 | 98.6 |
| Generation policy | Consistency | 26.8 | 32.7 | 98.5 |
| Common + Generation | Consistency | 27.1 | 32.8 | 98.6 |
| Candidate pool | Selection | Avg. | Exec. | |
| Common + Generation | First-valid | 24.8 | 99.2 | |
| Common + Generation | Learned | 25.3 | 99.3 | |
| Grounded task interface | Consistency | 27.2 | 99.3 | |
| Common policy | Consistency | 28.4 | 99.2 | |
| Generation policy | Consistency | 29.0 | 99.4 | |
| Common + Generation | Consistency | 30.1 | 99.5 |
Table 5. Selection results on SpreadsheetBench and Domain-Spreadsheet (%) with . Common + Generation samples two candidates per policy. The selected program is reviewed once for intent misalignment.
32.8% average accuracy, improving over first-valid by 2.1 percentage points on both metrics and over learned selection by 1.3 and 1.4 points, respectively. On Domain-Spreadsheet, average accuracy reaches 30.1%, compared with 24.8% for first-valid and 25.3% for learned selection.
Candidate Composition. Under consistency selection, mixing policies achieves the highest accuracy on both benchmarks. On SpreadsheetBench, the mixed pool reaches 27.1% hard accuracy, exceeding the common-only (26.6%) and generation-only (26.8%) pools. On Domain-Spreadsheet, it achieves 30.1% average accuracy, compared with 28.4% and 29.0%, respectively. These results highlight the benefits of combining complementary policies over single-policy sampling.
Oracle Upper Bound. Table 6 compares multi-candidate execution consistency with oracle selection under the same candidate pool. On SpreadsheetBench, the oracle reaches 39.7% average accuracy, leaving a 6.9 point gap over consistency selection. The gap is larger on Domain-Spreadsheet (10.6 points), indicating that further improvements in candidate selection remain possible.
| Benchmark | Selected | Oracle | Gap |
| SpreadsheetBench | 32.8 | 39.7 | 6.9 |
| Domain-Spreadsheet | 30.1 | 40.7 | 10.6 |
Table 6. Oracle bounds for Common + Generation () in average accuracy (%). Oracle selects one program per task using references. Gap measures the difference from consistency selection in percentage points.
Conclusion
We presented SheetAlign, a spreadsheet manipulation framework combining grounded task interaction, stage-isolated policy learning, and execution consistency. On SpreadsheetBench, our grounded task interface improves Qwen3-4B’s average accuracy from 13.5% to 29.2% without training. Stage-isolated learning further increases accuracy to 30.4%, while execution consistency raises it to 32.8%. On Domain-Spreadsheet, SheetAlign achieves 30.1% average accuracy, compared with 17.2% for the outcome-based RL baseline. These results demonstrate the effectiveness of combining task grounding, stage-isolated policy learning, and consistency selection for spreadsheet agents.
Limitations
Our study focuses on spreadsheet manipulation using compact open-weight LLMs with Python and openpyxl. Tasks involving specialized Excel features or external integrations may require additional tools. Task-guided inspection uses bounded workbook observations, which may need further adaptation for exceptionally large or complex workbooks. Moreover, execution consistency introduces additional inference costs by generating multiple candidates, creating a trade-off between computational efficiency and solution reliability.
Ethics Statement
All datasets and models used in this paper are publicly available, and their usage strictly adheres to the corresponding licenses and terms of use.
Method Details
Grounding and Execution
Structured task description. Before code generation, the model receives the instruction, target locations, and a compact workbook preview. It returns six fields: where, from, what, write_mode, risks, and api_needs. These fields specify where to edit, what workbook information to inspect, what operation is requested, how the result should be written, and any relevant risks or API requirements.
For example, consider a task that asks to fix a circular formula in B323. The workbook contains
B320 = 605
B323 = =IF(A323="E","",MAX(B$7:B377)+1)
where the range B7:B377 includes B323 itself. A corresponding task description can be:
{
"where": "B323",
"from": "A323 and previous values in column B",
"what": "fix the circular MAX formula",
"write_mode": "formulas",
"risks": ["circular reference"],
"api_needs": ["cell access", "formula
assignment"]
}
The harness then exposes only the workbook context needed for the task, such as the target cell, nearby values, relevant formulas, headers, and merged regions. This grounds generation in the current workbook state.
Observation construction. The harness builds a compact view of the task-relevant workbook region, including nearby cells, formulas, headers, merged ranges, and cell types. Formula expressions are preserved during inspection so that the generator can reason over the actual workbook logic.
API lookup. The api_needs field is used to retrieve a small set of relevant openpyxl references, which are provided to the generator together with the workbook observations. The lookup is deterministic and based on task-specific API keywords.
Diagnostic repair.
The harness executes each generated program and checks for clear implementation failures. When the program fails to run or produces an obviously invalid workbook, the harness starts a diagnostic repair. For each repair, the model receives the current program, a concise diagnostic, the user instruction, and the relevant workbook and API context. The model returns a complete corrected program. The harness then reruns it from the original workbook. Repair stops when no triggering problem remains or after two repair attempts.
Diagnostic repair uses only problems observed during program execution and workbook checking. This mechanism provides a fast correction path for implementation failures. It complements the intent misalignment review policy introduced in § 3.2, which targets programs that execute successfully but may still fail to reflect the user’s intent.
Repair prompt. The repair prompt is organized around the current failure and the information needed to correct it:
Goal. Repair the current program using the diagnostic. Correct the identified issue, preserve unrelated logic, and follow the same spreadsheet-editing constraints used during generation.
Context. The prompt provides the user instruction, target locations, current program, diagnostic message, focused workbook observations, and relevant openpyxl references.
Expected output. Return one complete corrected Python program that can be executed directly by the harness.
Policy Optimization and Training Settings
GRPO optimization. For each training prompt , group relative policy optimization (GRPO) (Shao et al., 2024) samples responses and assigns each response a reward . We compute the group-relative advantage as
where and are the mean and standard deviation of rewards for responses sampled from the same prompt.
For token in response , the policy ratio is
where denotes the policy used to generate the training responses. We define the clipped ratio
and optimize
The response-level advantage is shared across tokens in the same response. Our implementation does not add a separate KL regularization term.
LoRA. We use low-rank adaptation (LoRA) [8] for parameter-efficient training. Each trained policy uses its own LoRA adapter, while the backbone parameters remain fixed. Generation policy and review policy training start from the same trained policy and proceed independently.
Training settings. Table 7 reports the shared optimization settings for generation policy and review policy training. Stage-specific data construction is described below.
| Setting | Value |
| Optimizer updates | 40 |
| Prompts per update | 4 |
| Responses per prompt () | 4 |
| Learning rate | |
| LoRA rank | 16 |
| LoRA scaling parameter | 32 |
| LoRA dropout | 0 |
| Prompt token limit | 12,288 |
| Response token limit | 12,288 |
| Sampling temperature | 0.8 |
| Top- | 0.95 |
| Checkpoint interval | 10 updates |
Table 7. Optimization settings for generation policy and review policy training.
Initial training data. The common starting policy is trained from a pool of 1,200 spreadsheet tasks. We use 1,136 tasks for training and 64 for validation. After filtering prompts that exceed the context limit, 1,127 training tasks remain eligible for optimization.
Generation policy curriculum. Generation policy training starts from a 256-task candidate pool. We prioritize tasks whose sampled programs contain both a correct solution and an executable but incorrect one. These tasks expose cases where the generation policy can solve the problem but does not yet do so consistently. We interleave them with training examples from different spreadsheet operation types to maintain broader coverage. Repeated sampling intentionally concentrates training on these unstable cases.
Review policy training. Review policy training uses a 256-task pool with multiple executable programs per task. We include both correct and incorrect programs so that the review policy learns when to keep an existing program and when to revise it. Using multiple programs also exposes the review policy to different candidate solutions for the same task.
Reward Definitions
We use two reward designs. The first trains the common starting policy. The second adds edit-level progress for subsequent generation policy and review policy training.
Initial diagnostic reward. A task can be evaluated on multiple workbook instances. Let denote the average correctness across these instances, and let indicate whether all instances are correct. We summarize task correctness as
where , placing more weight on correctness across individual workbooks while retaining a signal for complete task success. The initial reward uses as its main signal, scaled by 0.82, with smaller contributions from execution success (0.10) and output quality (0.06). Executable but incorrect programs receive a small positive reward, while execution failures receive negative penalties ranging from to depending on the error type. This design prioritizes task correctness while still encouraging executable and well-formed programs.
Progress reward details. The implementation explicitly handles cases with no required or predicted edits to avoid undefined precision or recall. Reference workbooks in training data are used only for reward computation and evaluation and are not provided to the policy.
Execution Consistency Details
Candidate configuration. We use four candidates in the fixed order . The two candidates use the common starting policy for generation, while the two candidates use the separately trained generation policy. Each candidate independently executes the full harness.
Workbook fingerprints. Fingerprints are constructed from the stored contents of each output workbook. They record worksheet names and order, nonempty cell locations, types, values, and formula expressions, together with formatting and layout information such as merged cells, styles, and filters. Formulas are compared as stored expressions without recalculation. Fingerprinting is limited to 200,000 inspected cells and 64 MiB of decompressed workbook content. Outputs that cannot be fingerprinted are excluded from agreement voting.
Selection procedure. Candidates are grouped by matching fingerprints. We select from the largest group containing at least two candidates, breaking ties by the fixed candidate order. If no group meets this condition, we choose the first candidate with an inspectable output, or the first candidate if none is available. Selection also requires no additional review policy calls or learned selection model.
Additional Experimental Details
Training and Inference Settings
Training hyperparameters are provided in Table 7. For benchmark inference, we use a sampling temperature of 0.2 and a context window of 32,768 tokens. Workbook understanding is limited to 1,024 output tokens, and program execution has a timeout of 300 seconds. Generation and repair outputs are constrained by the remaining context budget. Each candidate is executed independently on the original input workbook, with at most two diagnostic repair attempts.
Prompt Templates
We present prompt templates used by the grounded task interface and policy learning stages. Placeholders are populated by the harness at runtime.
Shared System Instruction. All stages share the following system instruction.
You write compact, executable Python for
spreadsheet manipulation. Use openpyxl only.
Programs must generalize across workbook
instances of the same task.
Task-guided Workbook Inspection. The model first produces a structured task description to guide focused workbook inspection. The harness then uses this description to retrieve relevant workbook observations.
Analyze the spreadsheet task and return JSON
only.
Do not generate code or reasoning traces.
Figure 3. Training dynamics of generation and review policies. The panels show mean training reward and execution success across optimization steps.
Instruction: {instruction}
Instruction type: {instruction_type}
Target locations: {answer_position}
Workbook preview: {workbook_preview}
Return exactly:
{
"where": "target sheets and ranges",
"from": "source data to inspect",
"what": "required operations",
"write_mode": "static_values | formulas |
formatting | structure |
unknown",
"risks": [],
"api_needs": []
}
Include only task-relevant details.
Use empty lists when uncertain.
API Grounding. Based on the identified API requirements, the harness retrieves relevant openpyxl signatures, usage notes, and documentation snippets. These references are inserted into generation and diagnostic repair prompts.
Relevant openpyxl API references:
{api_name}{installed_signature}
Usage: {usage_note}
Documentation: {short_installed_docstring}
Canonical examples:
ws.cell(row=r, column=c).value = value
ws.delete_rows(idx=row_index, amount=1)
for row in ws.iter_rows(min_row=2):
values = [cell.value for cell in row]
Program Generation. The generation policy receives the task description, focused workbook observations, and API references to produce an executable transformation program.
Generate Python/openpyxl code for the requested
workbook transformation.
Return one Python code block defining:
transform(wb) -> None
Requirements:
- Do not load, save, or access workbook files.
- Use openpyxl and the Python standard library.
- Apply edits to the intended sheets and ranges.
- Preserve unrelated cells, formulas, and styles.
- Prefer static values for computed outputs;
use formulas when explicitly requested.
- Handle missing values, numeric strings, and
merged cells conservatively.
- Avoid unsupported openpyxl APIs.
- Generalize across instances of the same task.
Mode: {mode}
Instruction type: {instruction_type}
Target locations: {answer_position}
Instruction: {instruction}
Workbook preview: {workbook_preview}
Task description: {understanding_json}
Focused inspection: {focused_inspection}
API references: {api_references}
Execution and Diagnostic Repair. When execution or output checks detect a problem, the harness provides a concise diagnostic to the generation policy. The revised program is rerun from the original workbook, with at most two repair attempts.
Repair the current transform(wb) program using
the supplied diagnostic.
- Fix the identified issue without replanning.
- Preserve correct edits and unrelated logic.
- Address execution failures, invalid outputs,
or verification errors.
- For value outputs, avoid uncached formulas.
- Return one Python code block containing
the complete repaired transform(wb).
- Do not load or save workbooks.
Instruction: {instruction}
Target locations: {answer_position}
Diagnostic: {diagnostic_json}
Current program:
{current_code}
Task description: {understanding_json}
Workbook context: {focused_workbook_context}
API references: {api_references}
Review Policy. After candidate selection, the review policy performs one pass to check the selected program for intent misalignment. It may retain or revise the program based on the available input and output observations.
Review the program against the user's
instruction.
Successful execution does not guarantee
correctness.
Use only the supplied workbook observations.
Incomplete observations are not evidence of
errors.
Return one of:
DECISION: KEEP
- Provide a brief supporting reason.
DECISION: REVISE
- Explain the identified issue briefly.
- Return the complete corrected transform(wb).
Preserve valid edits and unrelated workbook
content.
Do not load, save, or access files.
Instruction: {instruction}
Current program:
{current_code}
Input/output observations:
{bounded_input_output_observations_json}
Excel Recalculation and Evaluation
Recalculation. Since openpyxl does not evaluate Excel formulas, we use Microsoft Excel on Windows to recalculate generated workbooks before final evaluation. The resulting workbooks are compared with reference workbooks using the official Spreadsheet-RL comparator.
Workbook comparison. The comparator evaluates the specified target regions after normalizing cell values. Numbers and numerical strings are rounded to two decimal places, dates are converted to Excel serial values, and times are normalized to HH:MM. Empty strings and missing values are treated as equivalent. Formula cells are compared using their recalculated values. A workbook instance is considered correct only if all target cells match the reference.
Recalculation scope. Excel recalculation is used only for final evaluation and does not affect policy training, diagnostic repair, or candidate selection.
Policy Training Dynamics
We examine the training dynamics of the generation and review policies in terms of mean training reward and execution success. The two policies optimize different stages of spreadsheet manipulation: generation constructs workbook transformations, while review addresses intent misalignment in executable programs.
Learned Selector
We train a lightweight candidate selector using Qwen3-4B-Thinking-2507 with LoRA. For each candidate program, the selector receives the user instruction, candidate code, and bounded observations of the input and output workbooks.
Training Labels. Candidate correctness is determined using the reference workbooks during training. For each training task, a candidate is labeled 1 only if its resulting workbooks are correct across all available instances, and 0 otherwise. Executable programs with incorrect outputs are also labeled 0. Samples with missing outputs, invalid reference comparisons, or observations exceeding the input budget are excluded. Reference workbooks are used only for label construction and are never included in selector inputs.
Training and Selection. We freeze the backbone model and optimize only the LoRA parameters using cross-entropy loss over the two label tokens, 0 and 1. At inference, each candidate is scored by the normalized probability
where and denote the logits of the two label tokens for candidate . The selector chooses the candidate with the highest score, without generating additional reasoning or code.
References
- [1]Xinyun Chen, Maxwell Lin, Nathanael Schärli, and Denny Zhou. 2024. Teaching large language models to self-debug. In ICLR. OpenReview.net.
- [2]Yibin Chen, Yifu Yuan, Zeyu Zhang, Yan Zheng, Jinyi Liu, Fei Ni, Jianye Hao, Hangyu Mao, and Fuzheng Zhang. 2025. Sheetagent: Towards a generalist agent for spreadsheet reasoning and manipulation via large language models. In WWW, pages 158–177. ACM.arxiv.org/abs/2403.03636
- [3]Mingyue Cheng, Shuo Yu, Daoyu Wang, Qingchuan Li, Xiaoyu Tao, Qingyang Mao, Yitong Zhou, and Qi Liu. 2026. Tableclaw: An interactive and self-evolving agent for spreadsheet manipulation and table reasoning. CoRR, abs/2606.10316.arxiv.org/abs/2606.10316
- [4]Banghao Chi, Yining Xie, Mingyuan Wu, Jingcheng Yang, Jize Jiang, Zhaoheng Li, Shengyi Qian, Minjia Zhang, Klara Nahrstedt, Rui Hou, Xiangjun Fan, and Hanchao Yu. 2026. Spreadsheet-rl: Advancing large language model agents on realistic spreadsheet tasks via reinforcement learning. CoRR, abs/2605.22642.arxiv.org/abs/2605.22642
- [5]DeepSeek-AI. 2025. Deepseek-v3.2: Pushing the frontier of open large language models. CoRR, abs/2512.02556.arxiv.org/abs/2512.02556
- [6]Haoyu Dong, Jianbo Zhao, Yuzhang Tian, Junyu Xiong, Mengyu Zhou, Yun Lin, José Cambronero, Yeye He, Shi Han, and Dongmei Zhang. 2024. Encoding spreadsheets for large language models. In EMNLP, pages 20728–20748.DOI
- [7]Ahmed El-Kishky, Alexander Wei, Andre Saraiva, Borys Minaiev, Daniel Selsam, David Dohan, Francis Song, Hunter Lightman, Ignasi Clavera Gilaberte, Jakub Pachocki, Jerry Tworek, Lorenz Kuhn, Lukasz Kaiser, Mark Chen, Max Schwarzer, Mostafa Rohanejad, Nat McAleese, o3 contributors, Oleg Mürk, and 5 others. 2025. Competitive programming with large reasoning models. CoRR, abs/2502.06807.
- [8]Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuntian Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. Lora: Low-rank adaptation of large language models. In ICLR. OpenReview.net.arxiv.org/abs/2106.09685
- [9]Aayush Kumar, Avik Dutta, Sumit Gulwani, Gustavo Soares, Advait Sarkar, and Emerson R. Murphy-Hill. 2026. Plans work in mysterious ways: Evaluating a plan mode for spreadsheet agents. CoRR, abs/2607.23670.arxiv.org/abs/2607.23670
- [10]Aviral Kumar, Vincent Zhuang, Rishabh Agarwal, Yi Su, John D. Co-Reyes, Avi Singh, Kate Baumli, Shariq Iqbal, Colton Bishop, Rebecca Roelofs, Lei M. Zhang, Kay McKinney, Disha Shrivastava, Cosmin Paduraru, George Tucker, Doina Precup, Feryal M. P. Behbahani, and Aleksandra Faust. 2025. Training language models to self-correct via reinforcement learning. In ICLR. OpenReview.net.arxiv.org/abs/2409.12917
- [11]Srivatsa Kundurthy, Clara Na, Colton Moraine, Anoushka Mohta, Case Winter, George Fang, John Ling, Emma Strubell, and Zach Kirshner. 2026. Bluefin: Benchmarking LLM agents on financial spreadsheets. CoRR, abs/2605.30907.arxiv.org/abs/2605.30907
- [12]Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In SOSP, pages 611–626. ACM.arxiv.org/abs/2309.06180
- [13]Hung Le, Yue Wang, Akhilesh Deepak Gotmare, Silvio Savarese, and Steven Chu-Hong Hoi. 2022. Coderl: Mastering code generation through pretrained models and deep reinforcement learning. In NeurIPS.arxiv.org/abs/2207.01780
- [14]Boyan Li, Ou Ocean Kun Hei, Yue Yu, and Yuyu Luo. 2026. DPC: training-free text-to-sql candidate selection via dual-paradigm consistency. In ACL, pages 6897–6913. Association for Computational Linguistics.
- [15]Hongxin Li, Jingran Su, Yuntao Chen, Qing Li, and Zhaoxiang Zhang. 2023. Sheetcopilot: Bringing software productivity to the next level through large language models. In NeurIPS.arxiv.org/abs/2305.19308
- [16]Zeyao Ma, Bohan Zhang, Jing Zhang, Jifan Yu, Xiaokang Zhang, Xiaohan Zhang, Sijia Luo, Xi Wang, and Jie Tang. 2024. Spreadsheetbench: Towards challenging real world spreadsheet manipulation. In NeurIPS.arxiv.org/abs/2406.14991
- [17]OpenAI. 2024. Gpt-4o system card. CoRR, abs/2410.21276.
- [18]Justin Payan, Swaroop Mishra, Mukul Singh, Carina Negreanu, Christian Pölitz, Chitta Baral, Subhro Roy, Rasika Chakravarthy, Benjamin Van Durme, and Elnaz Nouri. 2023. Instructexcel: A benchmark for natural language instruction in excel. In EMNLP, volume EMNLP 2023, pages 4026–4043.arxiv.org/abs/2310.14495


