CoworkEnv: Synthesizing Training Environments for Long-Horizon Cowork Agents

Code | Models | Lessons

Introduction

AI coworkers are expected to carry out everyday work through sustained interaction with users, tools, and a changing workspace. Resolving a customer-support request, for example, can require consulting account history, locating a relevant procedure, and updating the correct ticket while respecting access restrictions. Such workflows combine multi-turn communication, information gathering, tool coordination, and decisions about when to act [57, 69, 70]. Recent coworker models and benchmarks make these capabilities explicit targets for agent development [9, 36, 59, 71]. Developing compact models for this setting requires training experience that captures these dependencies.

Constructing that experience is a central challenge. The information needed to complete a workflow is often distributed across documents, service records, and intermediate environment state. Synthesizing a useful task therefore requires the request, the available evidence, and the permitted actions to fit together. Records referring to the same customer must agree across services; tools must expose the information needed to proceed and implement the expected state changes; and grading checks must assess relevant actions and outcomes. A plausible final response alone cannot establish that the intended update occurred or that a prohibited action was avoided. These requirements make executable environment construction and verification integral to data synthesis, as also emphasized in coworker evaluation and computer-use training [36, 58].

Open agent-training efforts have begun to expose the data choices behind capable systems. Nemotron-Terminal and SERA study synthetic tasks and demonstrations for terminal and coding agents, while OpenThoughts-Agent investigates data curation across agentic tasks [43, 48, 53]. Occamy extends execution-grounded training to coworker workloads [9]. Building on these efforts, we focus on a practical question: how can targeted coworker capabilities be translated into executable training tasks that support both imitation and learning from interaction?

We present CoworkEnv, a capability-guided framework for synthesizing executable training environments for AI coworkers. Developer-selected capability requirements guide the workflow scope of task construction. A workflow specification is implemented as a request, initial service records, tool interfaces, and grading checks. We describe two forms of task variation: composing business workflows across different services, and creating safety scenarios with distinct permitted and prohibited actions. Resolving a customer-support issue, for example, connects CRM context with a helpdesk ticket and a knowledge-base procedure; investigating a payment-change request connects correspondence with financial history. In the documented safety-task family, generation agents draft task content and a separate grading role defines the execution checks. Teacher executions provide curated demonstrations for supervised fine-tuning (SFT), while student interactions yield rollouts and reward feedback for reinforcement learning (RL). This process separates the construction of a problem from the generation of attempts to solve it. Figure 1 shows how task construction, teacher execution, and student training fit together.

To assess the training utility of these resources, we fine-tune Qwen3-VL-8B-Instruct [7] to obtain PocketWorker-8B. SFT raises the mean score from 46.6 to 64.1 on Claw-Eval tasks, with a 31.9-point gain on Multi-turn tasks. A separate SFT-only application to ClawMark extends the evaluation to a second coworker benchmark. We also compare training configurations for GRPO, Dr.GRPO, GDPO, and GiGPO to examine advantage estimation and credit assignment. Training rewards improve over approximately three epochs on our constructed tasks, providing preliminary evidence that these environments support RL optimization.

Our contributions are:

• Capability-guided synthesis of executable coworker environments. We translate target capabilities into multi-service workflows with consistent user requests, service records, tool interfaces, and executable grading checks. Task variation spans service dependencies and business constraints, including safety scenarios that require investigation while avoiding prohibited actions. Teacher execution and trajectory curation produce SFT demonstrations, while the constructed environments support interactive RL training.

• Training utility and analysis. We evaluate these resources through SFT gains on Claw-Eval, an SFT-only extension to ClawMark, and comparisons of RL training configurations.

We will release our datasets and construction pipeline, sharing the methods and design choices behind task and environment construction, trajectory collection, and data curation. We hope this helps other researchers adapt the process to their own needs.

For academic researchers working with limited resources, advanced topics such as long-horizon agentic SFT and RL can be difficult to pursue with limited GPU access, infrastructure gaps, unfamiliar engineering pitfalls, and high API costs. Our Lessons bring together challenges we encountered, ongoing work, and possible future directions, covering how different harnesses affect training outcomes, training–inference mismatch across harnesses, credit assignment and RL optimization. We share these experiences to help others make progress under similar constraints and avoid common pitfalls.

Related Work

Related Work

Agent post-training. Open post-training efforts make training data and recipes available for study [28]. AgentTuning develops general agent abilities through instruction tuning, while Nemotron-Terminal, OpenThoughts-Agent, and SERA investigate task generation and trajectory curation for terminal, coding, and broader agentic behavior [43, 48, 53, 72]. Related recipes extend trajectory-based training to research, data analysis, and GUI workflows [12, 16, 46]. AgentGym-RL and RAGEN study learning through sequential interaction, while ToolRL examines reward design for tool use [45, 61, 63]. slime provides a post-training framework with customizable data generation for RL scaling [77]. Agent harnesses organize execution and interaction interfaces [25, 67], with LEGO-RL connecting native coding-agent harnesses to policy-gradient training [13]. Polar provides asynchronous rollouts over existing agent harnesses, reconstructing training trajectories from proxied model API calls [68]. Building on PPO and GRPO [51, 52], subsequent methods explore advantage and loss normalization, reward-component normalization, and finer-grained credit assignment [17, 30, 33, 62]. Our framework connects executable task construction, curated demonstrations, and interactive RL for compact coworkers.

Executable task synthesis and demonstration curation. Task synthesis couples generated requests with interaction settings and completion criteria. AgentGen and MATRIX construct environments or simulated scenarios [24, 56]; APIGen-MT links task blueprints to multi-turn interactions, and CUA-Gym jointly constructs instructions, environments, and rewards [44, 58]. CuES generates executable tasks by exploring existing tool environments [34]. Verification and refinement further shape training data: ToolACE verifies generated tool-use dialogues, while AgentRefine studies refinement-oriented demonstrations [18, 31]. Complementary approaches obtain training experience through environment models and adaptive generation: DreamGym and Qwen-AgentWorld simulate environment dynamics, PatchWorld induces executable world-model programs with counterexample-driven repair, and AgentEvolver combines task generation with experience-guided exploration [6, 10, 73, 79]. Concurrent work includes Cowork-Gym [78], which explores large-scale synthesis of interactive work environments. Our construction jointly specifies cross-service records, tools, and grading checks, then validates tasks and curates teacher executions for SFT while retaining environments for RL.

Coworker agents and evaluation. Occamy studies coworker specialization through execution-grounded data and staged post-training [9]. AppWorld, $\tau$-bench, TheAgentCompany, and AutomationBench evaluate multi-application work, user interaction, and workflow constraints [54, 57, 69, 70], while GDPval evaluates economically valuable professional tasks [42]. ClawMark emphasizes persistent workflows and evolving service state, and Claw-Eval evaluates task completion, safety, and robustness [36, 71]. AgentDojo and ToolEmu assess risks from prompt injection and agent actions [11, 49]; Terminal-Bench and CyberGym provide execution-based evaluation in terminal and software-security settings [37, 60]. We evaluate SFT on Claw-Eval and ClawMark and compare RL configurations on Claw-Eval to study learning for coworker workflows.

Method

CoworkEnv constructs executable coworker environments through the following stages: capability-guided task design, executable environment construction, programmatic validation and feedback repair, teacher execution and curation, and preparation for SFT or RL. The central unit is a task environment that brings together a request, the information and tools needed to act, and checks of the resulting execution. Teacher demonstrations provide supervision for imitation, while student interaction with the environment provides experience and rewards for RL.

Capability-Guided Task Synthesis

From capabilities to task specifications. We translate target capabilities into a coverage plan specifying service combinations, intended difficulty, and business scenarios. Scenarios define selection conditions, distractor records, and permitted actions; a fixed reference date anchors time-sensitive requirements. An LLM receives the plan together with service catalogs, an output-format specification, and example tasks. It generates the user request, initial service records, structured completion checks, and prohibited actions. The catalogs describe available tools and record formats, guiding generation toward reusable service implementations.

We represent a task as

where $u_i$ is the request, $s_i^0$ the initial state, $\mathcal{A}_i$ the tool interfaces, and $\mathcal{G}_i$ the grading checks. Programs instantiate the specification: service configurations are populated from a registry, and structured completion checks and prohibited actions are inserted into a fixed grader template. Shared services expose the generated records and support state-changing actions. Task variation thus changes the scenario, records, and checks while reusing the execution and grading implementations. Appendix § 12 gives the plan format and template structure.

Linking information across services. Workflow construction distributes related information across services. An escalation task, for example, combines ticket creation times with customer tiers to select enterprise-account tickets open for more than 48 hours, then retrieves the account manager’s address for an email. A recent high-priority ticket is a distractor: it appears urgent but fails the age condition. Grading checks reference entities and actions from the same specification, using tool arguments and service audit records to check execution. This connects the requested goal, the evidence available through tools, and the scoring conditions.

Safety scenarios introduce constraints on acting on retrieved information. A payment-change investigation combines a suspicious email, legitimate correspondence, and transaction history. Completion checks reward evidence gathering, while designated submission or messaging actions activate a safety gate. Under the Generator–Discriminator design, the Generator drafts the request and records; the Discriminator receives the records and a note describing the safety scenario in a separate context, and specifies completion checks and forbidden actions. Appendices § 11 and § 12 describe the role inputs and workflow examples.

Validation and Feedback Repair

Checking generated specifications. Programmatic checks complement the generation instructions. They verify that the specification can be parsed, that selected services are registered and have initial records, that completion weights sum to one, and that required and prohibited tool sets do not conflict. A request-level leakage check rejects designated answer identifiers exposed in the user request, preserving the need to discover them through interaction.

Teacher Execution and Data Curation

Checking grading behavior. A grading consistency gate evaluates synthetic tool-call and audit records against the declared checks. A record satisfying the completion conditions must receive full completion and safety scores; an empty record must receive zero completion; adding a prohibited call must set the safety score to zero.

Feedback and regeneration. Validation failures are returned to the generator as diagnostic feedback. The generator produces a revised full specification, which passes through the same validation pipeline. Only candidates satisfying both specification checks and grading-consistency tests are packaged with their initial records, service configuration, and grading code for execution. Appendix § 12 gives the retry budget and test construction.

Figure 1 illustrates the task components and the teacher-demonstration path. To characterize the construction libraries, we group their tasks into six descriptive families: business workflows, multimodal work, knowledge and content, interactive advice, engineering, and security and compliance. Figure 2 summarizes service coverage and task composition.

Teacher Execution and Data Curation

Collecting interactions. A teacher executes tasks through a Claude Code adapter [2] in Harbor [21]. The runtime loads the task’s initial service records and manages service startup and reset for interactive execution. Starting from the request and initial state, the teacher selects actions and receives observations from the configured services. One attempt yields

where $j$ indexes attempts, $a_t$ is an assistant response or tool action, and $o_t$ is its observation; a final response need not have a following observation. The grader scores the execution, and the record preserves tool arguments, returned observations, and task and trial identifiers. Repeated attempts provide both alternative demonstrations and a task-level view of teacher performance. A separate path collects dialogue demonstrations on our independently constructed multi-turn tasks with a simulated user; Appendix § 13 describes their preparation. All training tasks are constructed independently of the benchmark test sets.

Selecting tasks and demonstrations. Task selection uses mean teacher completion to identify consistently completed tasks, tasks with very low completion, and an intermediate group. We retain qualifying demonstrations from the intermediate group, limit consistently completed tasks to one demonstration per normalized task prefix, and set aside the lowest-completion group. This reduces repeated supervision from easy tasks while retaining multiple executions of other tasks. When merging collections, we deduplicate by task and trial identifier so that each recorded execution enters the pool once. Appendix § 13 specifies the grouping and export rules.

Demonstration selection proceeds in two stages. Export first retains trajectories meeting source-specific reward thresholds. Conversion then removes records that cannot yield usable user and assistant messages, along with sequences that are too short, contain too many tool calls, or exceed the whole-example character budget. These checks select the interactions used for student supervision.

Preparing training inputs. The retained executions are serialized as user, assistant, and tool messages with structured tool calls and a standardized system instruction. Long tool observations are shortened to a prefix during conversion; whole examples exceeding the character budget are discarded.

Appendix § 13 gives the thresholds, processing order, and conversion statistics.

Training Interfaces for SFT and RL

We used slime [77] and Polar [68] during development of our training pipeline.

SFT from teacher demonstrations. The SFT interface supplies the curated message sequences to the student’s chat template. We adapt Qwen3-VL-8B-Instruct [7] with LoRA [23], supervising assistant responses and tool calls while using user messages and tool observations as context. For a demonstration set $D_{\mathrm{SFT}}$, serialized tokens $x_{\tau,\ell}$, and assistant-target mask $m_{\tau,\ell}$, the objective is

Execution rewards guide demonstration selection; the loss assigns equal weight to supervised tokens.

RL through student interaction. The RL interface supplies references to selected task definitions. Each reference loads the request, initial service records, tools, and grader. The student generates its own trajectory through online tool interaction, and the grader computes rewards from that execution’s calls and service audit records. Teacher performance can inform task selection, while the learning signal comes from the student’s behavior. Missing rewards abort the affected sample, and incomplete or infrastructure-failed rollout groups are rejected and retried before advantage estimation.

The two interfaces reuse task definitions, service implementations, and grading mechanisms, with selection tailored to demonstration learning or online interaction. This makes environment construction a shared foundation for both forms of adaptation. § 5 reports the experimental results; Appendix § 9 provides the training configurations.

Experiments

Experimental Setup

Models. We use PocketWorker for the compact models trained using CoworkEnv. The two 8B training branches use Qwen3-VL-8B-Instruct [7]. We evaluate the mixed-data model, PocketWorker-8B (ClawMark SFT), on ClawMark. For Claw-Eval, we evaluate the distinct 8B SFT checkpoint PocketWorker-8B (Claw-Eval SFT) against the base model under the same evaluation configuration. We also evaluate Qwen3-VL-4B-Instruct and its dialogue-only SFT checkpoint selected after two epochs.

Benchmarks and Metrics. ClawMark contains 100 tasks across 13 professional domains, involving multiple turns and evolving service state [36]. For Claw-Eval, we use the 161 General and 38 Multi-turn tasks, totaling 199 [71]. The primary metric is the mean task score. For task scores $s_i\in[0,1]$, we report $100N^{-1}\sum_{i=1}^{N}s_i$ on a 0–100 scale. On ClawMark, Table 2 reports mean task scores on 100 tasks. For the main Claw-Eval base and SFT results, we use three trials per task under the same evaluation configuration and report mean task scores. The data-scale ablation in § 5.5 (Table 4) reuses these base and full-data SFT mean scores. For tasks evaluated in three trials, Pass@3 and Pass$^{3}$ report the percentages of tasks with at least one and all three trial scores reaching 0.75, respectively. The threshold is applied to each trial before counting qualifying tasks. Pass$^{k}$ in $\tau$-bench similarly measures reliability by requiring success across repeated trials [70]. Table 1 uses Overall for General plus Multi-turn.

Model	General $n = 161$ Score	General $n = 161$ Pass@3	General $n = 161$ Pass$^{3}$	Multi-turn $n = 38$ Score	Multi-turn $n = 38$ Pass@3	Multi-turn $n = 38$ Pass$^{3}$	Overall $n = 199$ Score	Overall $n = 199$ Pass@3	Overall $n = 199$ Pass$^{3}$
Claude Opus 4.6	80.6	80.8	70.8	79.6	89.5	68.4	80.4	82.4	70.4
Claude Sonnet 4.6	81.3	81.4	68.3	81.9	89.5	65.8	81.4	82.9	67.8
GPT 5.4	78.3	75.8	60.2	79.0	89.5	60.5	78.4	78.4	60.3
Gemini 3.1 Pro	76.6	80.8	55.9	80.2	92.1	65.8	77.3	82.9	57.8
MiMo V2 Pro	76.0	72.7	57.1	81.0	92.1	60.5	77.0	76.4	57.8
Qwen 3.5 397A17B	73.8	70.8	57.8	75.6	76.3	52.6	74.2	71.9	56.8
GLM 5 Turbo	73.8	73.9	57.1	77.2	84.2	50.0	74.4	75.9	55.8
GLM 5V Turbo	73.2	73.3	52.8	77.4	86.8	57.9	74.0	75.9	53.8
Gemini 3 Flash	71.0	67.1	48.4	77.5	84.2	52.6	72.3	70.4	49.2
MiniMax M2.7	71.8	72.0	49.7	75.9	84.2	44.7	72.6	74.4	48.7
MiMo V2 Omni	74.1	75.2	52.2	65.4	63.2	15.8	72.4	72.9	45.2
DeepSeek V3.2	68.3	71.4	42.2	64.0	60.5	31.6	67.5	69.3	40.2
Kimi K2.5	66.6	67.1	36.6	75.4	76.3	39.5	68.3	68.8	37.2
Nemotron 3 Super	41.7	34.8	6.8	56.2	13.2	0.0	44.4	30.7	5.5
Qwen3-VL-8B-Instruct	46.7	27.3	23.6	45.9	34.2	13.2	46.6	28.6	21.6
PocketWorker-8B (SFT)	60.9 (+14.2)	44.1 (+16.8)	34.2 (+10.6)	77.8 (+31.9)	78.9 (+44.7)	50.0 (+36.8)	64.1 (+17.5)	50.8 (+22.1)	37.2 (+15.6)
Qwen3-VL-4B-Instruct	28.7	14.3	11.2	56.1	42.1	10.5	33.9	19.6	11.1
PocketWorker-4B (SFT)	36.6 (+7.9)	13.7 (−0.6)	10.6 (−0.6)	66.0 (+9.9)	60.5 (+18.4)	36.8 (+26.3)	42.2 (+8.3)	22.6 (+3.0)	15.6 (+4.5)

Model	Tasks	Score $\uparrow$
GPT-5.4	100	56.7
Claude Sonnet 4.6	100	53.1
Qwen3.6 Plus	100	48.7
Gemini 3.1 Pro Preview	100	38.9
MiniMax M2.7	100	33.0
Kimi K2.5	100	56.0
GPT-5.4-mini	100	28.4
Qwen3-VL-8B-Instruct	100	11.8
PocketWorker-8B (SFT)	100	19.4<br>(+7.6)

SFT data	General $n = 161$	Multi-turn $n = 38$	Overall $n = 199$
Base (no SFT)	46.7	45.9	46.6
20%	50.1 (+3.4)	53.6 (+7.8)	50.8 (+4.2)
50%	53.4 (+6.7)	64.9 (+19.0)	55.6 (+9.0)
100% (full)	60.9 (+14.2)	77.8 (+31.9)	64.1 (+17.5)

Published reference models. We include 14 published Claw-Eval reference models and seven ClawMark website reference models, including strong proprietary systems and larger open-weight models. Claw-Eval scores are reproduced from v3 of the paper [71]. ClawMark reference scores use the website’s Run 1 over 100 tasks [15]. Appendix § 14 records the sources and evaluation settings.

Claw-Eval Results

The 8B results in Table 1 compare PocketWorker-8B (Claw-Eval SFT) with the base model under the same evaluation configuration. On all Multi-turn tasks, the SFT model scores 77.8, compared with 45.9 for the base, a difference of 31.9 points. On General tasks, the corresponding scores are 60.9 and 46.7, a difference of 14.2 points.

Across General and Multi-turn tasks, the reported Overall score rises from 46.6 to 64.1, a gain of 17.5 points. The SFT model obtains Pass@3 / Pass$^{3}$ of 78.9 / 50.0 on Multi-turn and 44.1 / 34.2 on General. The combined rates are 50.8 / 37.2, compared with 28.6 / 21.6 for the base model. These metrics distinguish reaching the pass threshold at least once from doing so consistently across all three trials.

Compact-model specialization. The Multi-turn result motivates testing whether targeted coworker training can narrow the gap to larger general models. Our 8B model’s 77.8 lies within the range of the published frontier-model scores, while its improvement of 31.9 points measures the change from our base under the same evaluation configuration. The reference set includes Nemotron 3 Super, Kimi K2.5, and Qwen3.5-397B-A17B, whose reported total/active parameter counts are 120B/12B, 1T/32B, and 397B/17B, respectively (Appendix § 14). A claim of outperforming a particular reference model requires aligned evaluation settings. The 4B results extend this comparison to a smaller model: SFT raises General from 28.7 to 36.6 and Multi-turn from 56.1 to 66.0, gains of 7.9 and 9.9 points, respectively. § 5.5 presents the 8B data-scale ablation (Table 4).

ClawMark Results

We extend CoworkEnv and its SFT pipeline to ClawMark, a second coworker benchmark. As shown in Table 2, PocketWorker-8B (ClawMark SFT) achieves a mean score of 19.4, compared with 11.8 for the base model, a gain of 7.6 points. Together with the Claw-Eval results, this SFT-only extension supports the applicability of our framework across different coworker workflow settings.

Comparison of RL Training Methods

Benchmark results. Starting from a shared checkpoint obtained through supervised warm-up, we train GRPO [52], Dr.GRPO [33], GDPO [30], and GiGPO [17] on our constructed RL tasks for approximately three epochs. Method-specific configurations are described in Appendix § 10. Table 3 reports the benchmark results of the warm-up-plus-RL pipelines. Relative to the Qwen3-VL-8B-Instruct baseline, the reported Overall scores increase from 46.6 to 56.6 with GRPO and 57.2 with GDPO, differences of 10.0 and 10.6 points. The largest differences occur on Multi-turn tasks: scores increase from 45.9 to 72.5 with GRPO and 74.9 with GDPO, corresponding to 26.7 and 29.0 points. General scores increase from 46.7 to 52.8 and 53.0, differences of 6.1 and 6.3 points, respectively. These aggregate comparisons show the largest gains in multi-turn interaction. Different reward-processing, advantage-estimation, and credit-assignment strategies exhibit distinct training dynamics and performance profiles across task categories. Both GRPO and GDPO achieve their largest gains on Multi-turn tasks, with GDPO attaining the higher score, whereas GRPO performs better on Multimodal tasks. These observations suggest that the benefits of different RL training configurations vary with the capabilities required by each task category.

Training condition	General $n = 161$	Multi-turn $n = 38$	Overall $n = 199$	Multimodal $n = 101$	All tasks $n = 300$
Qwen3-VL-8B-Instruct	46.7	45.9	46.6	20.7	37.9
GRPO	52.8 (+6.1)	72.5 (+26.7)	56.6 (+10.0)	22.1 (+1.4)	45.0 (+7.1)
GDPO	53.0 (+6.3)	74.9 (+29.0)	57.2 (+10.6)	21.0 (+0.3)	45.0 (+7.1)
Dr.GRPO	51.2 (+4.6)	67.1 (+21.2)	54.3 (+7.7)	21.0 (+0.3)	43.1 (+5.2)
GiGPO	59.7 (+13.0)	64.8 (+18.9)	60.7 (+14.1)	21.0 (+0.2)	47.3 (+9.4)

Training dynamics. Figure 3 in Appendix § 10 shows training reward, policy loss, and response length in separate panels for each method. The horizontal axis measures cumulative processed samples, including repeated sampling across epochs. Training rewards increase over the course of optimization, with different trajectories across methods, providing evidence that our constructed environments support policy optimization with multiple RL methods.

Ablation Study

Training-data scale. Table 4 compares the 8B base model with SFT using 20%, 50%, and 100% of the training data. For each reduced-data condition, we randomly sample the specified fraction separately within each SFT stage. Overall scores increase from 46.6 for the base model to 50.8, 55.6, and 64.1, respectively. General and Multi-turn scores also increase across these data fractions, with full-data SFT achieving the highest scores. These results support the utility of increasing the amount of available coworker supervision.

Conclusion

We present CoworkEnv, a capability-guided framework for synthesizing executable training environments for AI coworkers. By aligning user requests, service states, tool interfaces, and grading checks, our pipeline supports both curated demonstrations for SFT and interactive training with RL. Experiments show SFT gains on Claw-Eval, while an SFT-only extension to ClawMark supports the framework’s applicability across coworker benchmarks. Supervised warm-up followed by RL improves Multi-turn performance over the base model, and comparisons of reward processing, advantage estimation, and credit assignment reveal distinct training dynamics and task-dependent performance profiles.

Limitations

Execution scores measure the properties encoded by the graders. They can miss errors outside those checks, while teacher-success filters can favor redundant demonstrations or discard useful recovery behavior. The role of unsuccessful trajectories also deserves study: work on iterative skill evolution reports that feedback conditions containing failures can produce useful skill updates [32]. Regenerating the original teacher trajectories may require access to a proprietary model service.

Ethical Considerations

CoworkEnv is designed for research in executable, simulated work environments rather than deployment in real-world organizations. Nevertheless, agents trained on synthetic workflows may perform incorrect or unauthorized actions if deployed without appropriate safeguards.

Training Configurations

Claw-Eval SFT

The final Claw-Eval SFT stage for the 8B model uses a per-device batch size of one, eight gradient accumulation steps, a 4,096-token maximum sequence length, and LoRA [23] rank 32, scaling factor 64, and dropout 0.05. The 4B SFT result in Table 1 uses a dialogue-only checkpoint selected after two epochs.

ClawMark SFT

We trained PocketWorker-8B (ClawMark SFT) for three epochs (354 optimizer updates) with a configured learning rate of $2 \times10^{-4}$, LoRA rank 32, scaling factor 64, and dropout 0.05. Training used a per-device batch size of one, eight gradient accumulation steps, and a maximum sequence length of 4,096 tokens.

RL Training

RL method configurations and comparison scope. Figure 3 compares training dynamics for GRPO, Dr.GRPO, GDPO, and GiGPO on our constructed task set. All four methods start from the same supervised warm-up checkpoint and train for approximately three epochs. Table 3 reports the final benchmark results for all four methods. The method-specific choices are described below. The GRPO baseline [52] centers the scalar reward within each group of eight rollouts and normalizes by the group standard deviation with a denominator floor of 0.05. Zero-variance groups within numerical tolerance receive zero advantages and remain valid groups. This normalization rule is separate from rejection of invalid rollouts.

Dr.GRPO. Following Liu et al. [ [33], our implementation changes both advantage normalization and loss normalization. Advantages are mean-centered without division by the reward standard deviation. The masked token losses are summed and divided by a fixed normalizer of 1,536, corresponding to the per-turn generation limit of 1,536 tokens. The comparison with GRPO therefore varies both advantage and loss normalization.

GDPO. Our GDPO implementation normalizes each reward dimension within the rollout group, combines the resulting advantages, and applies batch-wise normalization [30]. Batch statistics use only valid groups. The GRPO baseline uses a joint scalar reward and a different batch size. The comparison varies reward processing and batch size together, so it evaluates the complete GDPO configuration.

GiGPO. GiGPO combines episode-level advantages with step-level advantages computed from groups of matching intermediate states [17]. We use a step-advantage weight of $\omega= 1.0$. The comparison evaluates this complete configuration without a matched $\omega= 0$ ablation. Policy-loss magnitudes use different normalizations across methods and should not be read as a common measure of policy quality.

Advantage-estimation context. Generalized advantage estimation uses value-function predictions to trade bias against variance in policy-gradient estimates [50]. VinePPO investigates credit assignment in language-model reasoning using Monte Carlo estimates [27], while Wei et al. [ [62] study terminal, delayed, and per-turn rewards for multi-turn agents. These works motivate separating reward granularity from the estimator that turns rewards into advantages. The configurations above use group-based advantage estimators, with GiGPO additionally grouping

matching intermediate states.

Other optimization choices. GSPO uses sequence-level importance ratios and clipping [74]. Score centering addresses drift caused by differences between training and inference engines [35]; this correction concerns training–inference mismatch, a different issue from subtracting a group’s mean reward. These methods provide context for the optimization design space, while the comparisons reported here comprise GRPO, Dr.GRPO, GDPO, and GiGPO.

A Constructed Safety Workflow

Generation and grading roles. The Generator–Discriminator design separates scenario drafting from check specification. The Generator receives a safety-scenario category and service combination, then drafts the request and initial records. The Discriminator receives the task and service identifiers, records, and a safety note describing the intended conflict, without the full request text. It specifies completion checks for investigation and a set of prohibited actions. Shared record identifiers connect these checks to the generated evidence. The resulting check and forbidden-tool sets are inserted into the grader template described in Appendix § 12.

Request and evidence. The instruction body of M133_safety_payment_change_refusal asks for a vendor banking update and payment-report submission. Its title describes verification without submission. The fixtures contain a spoofed request with a new account, legitimate correspondence identifying the established account, and historical transactions consistent with that account. A third mail message is an unrelated announcement. These records make the contradiction accessible through mail and finance tools. The example demonstrates how a task can pair a request to act with evidence relevant to withholding the action.

Grading. The grader checks five actions: opening the inbox (weight 0.15), reading the suspicious request (0.25), reading the legitimate vendor message (0.20), listing financial history (0.20), and inspecting relevant transactions (0.20). Calling finance_submit_report or gmail_send_message sets the safety factor to zero. The score is $r = \operatorname{safety}(0.8\ \text{completion} + 0.2\ \text{robustness})$. These tool-based checks measure evidence access and avoidance of designated actions; absence of a forbidden call alone does not establish semantic recognition of fraud or an explicit refusal in the final response.

Grading validation. The grading-consistency procedure exercises completion checks and the forbidden-tool gate using synthetic tool-call and audit records. Appendix § 12 describes the test cases and feedback-based regeneration procedure.

Task selection. The three teacher attempts for M133 each achieve completion 0.8, placing the task in the KEEP group under the thresholds in Appendix § 13.

Another workflow represented in training. The M1003_ticket_kb_resolution package specifies SSL certificate triage using helpdesk and knowledge-base services. The task package includes metadata, grading code, and an initial helpdesk ticket fixture. Its three retained mixed-data records have reward 0.84, message counts of 13, 9, and 10, and tool-call counts of 6, 4, and 5, respectively.

Constructing Multi-Service Workflows

Coverage plan and generation prompt. Each plan slot supplies seven fields: task_id identifies the output task; services selects service catalogs; category names the workflow category; difficulty specifies intended complexity; language sets the request language; mock_today fixes the reference date; and scenario describes the goal, selection conditions, distractors, and action constraints. The following abridged escalation slot uses a descriptive task identifier:

task_id: support_escalation
services: [crm, helpdesk, gmail]
category: ops
difficulty: hard
language: en
mock_today: "2026-03-10"
scenario: >-
  Find enterprise tickets open
  for more than 48 hours and email
  the account manager.
  Distractor: a high-priority ticket
  less than 48 hours old.

The generation prompt combines output-format requirements, catalogs for the selected services, example specifications, and the plan slot. Additional multi-service instructions ask the generator to distribute evidence across services and place the information needed to determine the answer in the initial records. Scenario variation changes selection conditions, information links, and permitted actions: a ticket-prioritization task can select by customer tier and update helpdesk records, while an escalation task adds an age threshold and retrieves a recipient from CRM for a mail action.

Template-based instantiation. The service registry supplies ports, health-check paths, and configuration metadata. Generated records populate the task’s fixtures. The grader builder serializes the completion checks and prohibited tools into a fixed code template. Each check specifies its kind, weight, and the tool arguments or audit fields it inspects. The following pseudocode summarizes the shared completion and safety calculations:

CHECKS = spec.completion_checks
FORBIDDEN = spec.safety_forbidden_tools

def components(calls, audit):
    completion = 0
    for c in CHECKS:
        ok = test(c, calls, audit)
        completion += c.weight * ok
    safety = 1
    for call in calls:
        if call.tool in FORBIDDEN:
            safety = 0
    return completion, safety

Task-specific targets and weights vary in the check data, while the evaluation routines are reused. Appendix § 11 gives a concrete reward composition using these components.

Service audit records and grading checks. An audit_present check names a service, an action_key, a record field, and accepted target values. The action key selects the service’s audit collection, linking a recorded operation to the fields examined by the grader. Two examples are:

• Helpdesk updates: helpdesk_update_ticket records an entry in updated_tickets; the check examines its ticket_id field for the target ticket.

• Outgoing mail: gmail_send_message records an entry in sent; the check examines its to field for the recipient associated with the generated customer records.

Design scope. The M1100–M1124 family contains 25 tasks constructed to cover multi-service workflows at two levels of intended complexity. Fifteen D1.5 tasks target three services, 8–12 steps, and distracting information; ten D2.5 tasks target four or five services, 12–20 steps, and temporal dependencies. These targets translate coverage requirements into concrete task specifications. They describe intended workflow complexity rather than measured teacher execution lengths. The construction record’s “Class C” mock-service label is distinct from the local “C” alias for Claw-Eval Multi-turn.

Request, fixtures, and checks. M1100 links a CRM customer marked at risk to a critical authentication ticket through a shared customer identifier. A knowledge-base article provides an API key rotation procedure. The instruction asks for a ticket update based on that procedure and prohibits CRM export. The package specifies eight exposed tool interfaces across the three services.

The grader uses six deterministic completion checks with weights 0.08, 0.15, 0.15, 0.12, 0.20, and 0.30, summing to one. The check vocabulary includes tool-call presence, record-specific arguments, and service audit records. The documented ticket-update check identifies an update to the intended ticket; it does not inspect the resolution text. The export tool activates a safety gate. These checks assess selected actions and records rather than every semantic requirement in the instruction.

Teacher executions. Three teacher executions with claude-opus-4-8 [4] obtain rewards of 0.88, 0.84, and 1.0, with completion scores of 0.85, 0.80, and 1.0 and safety scores of 1.0 throughout.

Validation and regeneration details. For each slot in the coverage plan, the generator has a budget of two attempts: an initial generation and at most one retry. Each candidate passes sequentially through JSON parsing, required/prohibited-tool conflict checks, completion-weight checks, service and initial-record validation, request-level identifier-leakage checks, and the grading-consistency gate. A failed check returns diagnostic feedback for the next attempt. The retry produces a complete specification and repeats the validation sequence. Candidates still failing after the second attempt are discarded; accepted candidates are packaged with their request, initial records, service configuration, and grader.

Synthetic grading tests. The gate imports the generated grader and constructs tool-call and service-audit records from the declared completion checks. Tool-presence checks receive the required calls; argument checks receive a declared acceptable value; audit checks receive records carrying the required fields. The complete synthetic record must yield completion and safety scores of 1. An empty call and audit record must yield completion 0. For tasks with prohibited tools, adding a prohibited call to the complete record must set safety to 0.

Task coverage. The 25 task identifiers in the M1100–M1124 family are absent from the official Claw-Eval 300-task benchmark.

Data Selection and Preprocessing

Workflow and dialogue preparation paths. Table 5 compares preparation of tool-workflow and multi-turn dialogue demonstrations, both collected on our own constructed tasks. Workflow demonstrations come from execution in the constructed service environments. Dialogue tasks supply a user persona and a simulator that produces subsequent user turns. Their LLM judge scores clarification and final-answer quality, with the task rubric identifying information the assistant should elicit. This is an instance of model-based dialogue evaluation [75].

The dialogue-specific builder processes teacher traces collected on our constructed dialogue tasks, selects the highest-scoring trial per task, removes simulator-specific user prefixes, and normalizes messages for student training. The mixed-pool converter prepares execution records as student chat messages using the rules below.

The collection filter groups tasks according to their mean recorded completion. Selection records cover the multi-service and safety collections, including the M1100–M1124 family and M133. The statistic summarizes repeated executions by the collecting teacher.

Task grouping. For task $i$, let $V_i$ contain the attempts with a readable numeric score $c_{ij}$. Missing or unparseable scores are excluded from $V_i$. The reader prefers the completion field and falls back to reward when the completion key is absent. The grouping statistic is

This is an average of potentially fractional scores, not the fraction of attempts exceeding a success threshold. The implemented labels are

Trial aggregation and task-level sampling. The exporter admits tasks labeled KEEP or TRIVIAL and sets aside UNSOLVABLE tasks. Collections from multiple execution directories are merged using the task name and trial identifier as a deduplication key; the first encountered copy of a trial is retained. For designated tasks re-executed after grading fixes, the replacement collection supersedes the earlier records. This prevents outdated and replacement records from contributing to the same export.

For TRIVIAL tasks, the SFT export keeps the first qualifying record per normalized task prefix. The prefix is the final path component up to its first underscore, so task-name variants sharing this prefix share a sampling allowance. Other admitted tasks can contribute multiple qualifying trials. The dialogue-specific selection path is summarized in Table 5.

Component	Generated tool workflows	Dialogue demonstrations
Task source	Coverage plans, generated specifications, and reusable services	Independently constructed dialogue scenarios and user personas
Interaction	An initial request followed by agent tool calls and service observations	A persona-based user simulator exchanges turns with the agent
Grading	Deterministic checks of tool calls, arguments, and audit records	An LLM judge evaluates clarification and final-answer quality
Preparation	Export execution records and convert them to student messages	Extract recorded dialogue trials, select demonstrations, and normalize messages

Reward-based selection at export. The mixed SFT pool applies a reward threshold of 0.5 before message conversion. Dedicated multi-service and safety exports use 0.80, the document-extraction export uses 0.999, and the Harbor-to-message export for ClawMark uses 0.3. These are distinct export configurations. The mixed-pool message converter applies the structural and length checks below without imposing an additional reward threshold.

Message conversion and structural checks. Execution records are converted into user, assistant, and tool messages, with assistant tool calls serialized as structured fields. A standardized system message is prepended. Records yielding no usable user–assistant interaction are removed. The converter also discards sequences with fewer than five message entries or more than 20 tool calls. Message counts include system, user, assistant, and tool entries; the minimum-length condition therefore counts messages, rather than dialogue rounds.

Each tool observation longer than 1,000 characters is shortened to its first 1,000 characters followed by ...[truncated]. After this preprocessing, the converter sums the characters in message content and JSON-serialized tool-call fields, including the truncation marker. It discards an entire example if this total exceeds 40,000 characters. Tool-observation shortening and whole-example rejection are separate operations. The trainer subsequently applies its configured token-length limit.

Conversion replay. Replaying the converter on the 1,809-record mixed SFT candidate snapshot yields 937 demonstrations. The reported rejection categories account for the other 872 records: 494 yield no usable user–assistant interaction, 142 have fewer than five messages, 140 exceed 20 tool calls, and 96 exceed the 40,000-character budget. These counts describe the converter’s decisions on this candidate snapshot. The output has mean recorded reward 0.862, mean message count 13.8, and mean tool-call count 6.8.

Relation to data selection and supervision. GALA selects representative samples through clustering and geometric distances to reduce redundancy in LLM self-training [66]. Our selection uses recorded completion, reward thresholds, and structural checks; measuring whether representation-based selection improves coverage remains a separate experiment. SoFT studies soft targets that balance learning from demonstrations with preserving the base distribution [26]. Our SFT objective in Equation (3) uses the observed assistant tokens as targets. This separates the choice of which trajectories to retain from the choice of supervision applied to those trajectories.

Published Baseline Sources and Protocols

Claw-Eval. The published reference rows in Table 1 reproduce the Score, Pass@3, and Pass³ columns from Table 3 of Ye et al. [ [71] (v3). Its Overall covers the 161 General and 38 Multi-turn tasks, totaling 199; the 101 Multimodal tasks are reported separately in the source’s Table 4. Published values are transcribed as reported rather than reconstructed from rounded group scores. The paper runs three trials with temperature zero and no injected service errors, enabling extended thinking where available. Gemini 3 Flash [19] judges open-ended General criteria; Claude Opus 4.6 [3] at temperature 0.7 provides the Multi-turn simulator and judge. Our task counts alone do not establish alignment with these settings. The local 8B base and SFT results use the same evaluation configuration, with three trials per task and the same per-trial pass threshold of 0.75. Scores and changes are rounded separately to one decimal place.

ClawMark. Table 2 includes seven reference models from the official ClawMark leaderboard [14]. Website reference scores use OpenClaw [55] Run 1 over the same 100 tasks [15]. We use the explicitly named Run 1 field for every model and task and report the official aggregate on a 0–100 scale to one decimal place. This reference protocol uses one trial per task. Our base and SFT results are shown separately, with 100 tasks per model.

Reference-model documentation. The reference systems in Table 1 and Table 2 are documented in the provider releases for GPT-5.4 [40], GPT-5.4 mini [41], Claude Opus 4.6 and Sonnet 4.6 [3 c], Gemini 3.1 Pro and Gemini 3 Flash [19, 20], MiMo-V2-Pro and MiMo-V2-Omni [64 b], MiniMax M2.7 [38], and Qwen3.6 Plus [1]. GLM-5-Turbo has official model documentation [Zhipu AI]; GLM-5V-Turbo and DeepSeek-V3.2 have technical reports [22 Liu et al.,] 2025a]. These references identify the models; the benchmark scores and protocols come from the Claw-Eval paper and ClawMark records cited above.

Model scale. The official model cards report 120B total / 12B active parameters for Nemotron 3 Super [8], 1T / 32B for Kimi K2.5 [39], and 397B / 17B for Qwen3.5-397B-A17B [47]. These are reported MoE parameter counts, not measured inference costs. We do not infer latency or cost savings from total-parameter ratios.