betaFeedback

WorksComputer Science

CHATDIFY: Automating End-to-End Dify Workflow Synthesis from Natural Language via Multi-Agent Collaboration

Abstract

The proliferation of Large Language Models has catalyzed powerful application development platforms like Dify, which enable users to build sophisticated LLM-powered workflows through modular node orchestration. However, manually configuring these workflows requires deep platform expertise and remains time-consuming and error-prone, creating a significant barrier for non-expert users. To address this challenge, we propose ChatDify, a novel multi-agent framework that automates end-to-end Dify workflow synthesis from natural language descriptions. ChatDify employs an Actor-Critic architecture comprising three specialized agents: the Planner Agent that decomposes user requests into structured execution plans, the Executor Agent that performs context-aware parameter synthesis and schema-augmented DSL compilation, and the Reviewer Agent that ensures reliability through hybrid verification with closed-loop refinement. We evaluate ChatDify on a curated benchmark of 100 production-inspired tasks across three state-of-the-art LLM backends, achieving 0.96, 0.91, and 0.75 success rates with Qwen-3.7, GLM-5.1, and DeepSeek-v4, respectively. These results demonstrate its robustness and practical applicability for lowering the technical barrier to creating sophisticated LLM applications.

Introduction

The proliferation of Large Language Models (LLMs) [19, 20, 2, 6, 4, 28, 26] has catalyzed the emergence of powerful application development platforms like Dify [5], Coze [3] and LangChain [18]. These platforms provide developers with modular tools to build sophisticated LLM-powered applications, such as Retrieval-Augmented Generation (RAG) systems [13, 9], chatbots [1], and complex agentic workflows. Dify, in particular, provides a user-friendly interface to facilitate the orchestration of various nodes including LLMs, knowledge bases, and custom tools into a coherent workflow structure, as illustrated in Figure 3a.

Despite the power of these platforms, a significant barrier to entry remains. Users must manually select, connect, and configure each node in the workflow, a process that demands both a clear understanding of the desired logic and familiarity with the platform’s specific functionalities. This manual configuration is time-consuming, error-prone, and can be particularly daunting for users without a technical background. As workflows grow in complexity, the cognitive load on developers increases, hindering rapid prototyping and iteration.

A natural remedy is to let users describe their goals in natural language and generate the workflow automatically. However, this is far from trivial, because a generated workflow must be correct at three levels, and each level can fail even when the previous one holds. (i) **Structural correctness: the request must be decomposed into a valid DAG, with appropriate nodes selected from Dify’s heterogeneous node catalog and arranged with correct branching and iteration logic. (ii) Configuration correctness: each node must follow Dify’s platform-specific schema, and its variable references (e.g., {{node_id.field}}) must point to type-compatible upstream outputs; general-purpose LLMs do not reliably master these details, and an error in one node can break all downstream nodes. (iii) Semantic correctness: a valid and executable workflow may still diverge from the user’s intent, and its open-ended outputs lack deterministic ground truth to check against.

To address these challenges, we propose CHATDIFY, a novel framework that enables users to synthesize and configure complete Dify workflows using only natural language. By simply describing their goal, users can leverage CHATDIFY to automatically construct the corresponding workflow graph, populate node parameters, and establish the necessary connections. This transforms the development process from a manual, designed drag-and-drop Dify workflow into a simple conversational experience, as illustrated in Figure 1.

Figure 1. CHATDIFY Automates Dify Workflow Synthesis via Natural Language, Advancing Beyond Manually Designed Drag-and-Drop Dify Workflows.

CHATDIFY orchestrates a multi-agent collaborative framework to synthesize executable Dify workflows from natural language descriptions (see Figure 2). This architecture comprises three specialized agents aligned with these three levels: the Planner Agent targets structural correctness by deconstructing high-level user requests into a structured, step-by-step execution plan; the Executor Agent targets configuration correctness through context-aware parameter synthesis and schema-augmented DSL compilation using a dedicated node knowledge base; and the Reviewer Agent verifies all three levels by combining static symbolic verification with dynamic runtime probing, driving a closed-loop refinement process. This integrated process substantially improves the reliability of generated workflows. The main contributions of this work are summarized as follows:

Figure 2. Overview of CHATDIFY Architecture: A multi-agent system for Dify workflow synthesis. The system employs an Actor-Critic paradigm where the Planner and Executor (Actors) synthesize the workflow, while the Reviewer (Critic) provides hybrid feedback for closed-loop refinement.

  • End-to-End NL-to-Dify Synthesis System. We present CHATDIFY, a framework for translating natural-language requirements into executable Dify workflows through workflow graph construction, node-level parameter synthesis, and schema-aware DSL compilation.

  • Actor-Critic Architecture with Hybrid Verification. We design a knowledge-grounded Planner–Executor–Reviewer pipeline where the Reviewer combines static checks over schema, topology, and variables with dynamic probing that pairs sandbox execution and an LLM-as-Judge semantic verifier.

  • Industrial Benchmark and Empirical Evidence. We curate 100 production-inspired Dify tasks across 8 categories. Across Qwen-3.7, GLM-5.1, and DeepSeek-v4, CHATDIFY achieves 0.96, 0.91, and 0.75 success rates, substantially outperforming ZeroShot and ZeroShot+Dynamic baselines.

Related Work

NL2Workflow

The task of generating executable workflows from natural language has advanced rapidly with LLMs. AutoFlow ([15]) combines in-context learning with supervised fine-tuning and reinforcement learning to synthesize workflow plans. WorkflowLLM ([8]) fine-tunes an LLM on large workflow corpora, and Action Engine ([7]) generates Function-as-a-Service (FaaS) pipelines with tool-augmented LLMs. In industrial settings, WorkTeam ([16]) divides workflow construction among supervisor, orchestrator, and filler agents, while Classifier-Augmented Generation (CAG) ([10]) builds IBM DataStage ETL workflows through classifier-guided stage prediction, edge prediction, and property inference. These methods either operate at a higher level of abstraction or target other platforms, and do not address Dify-specific node configuration. CHATDIFY instead synthesizes executable Dify workflows with node-level parameters, explicit data-flow connections, and cross-node variable references, and verifies each candidate through static checks and sandboxed execution.

LLM Agents

LLM-powered agents have emerged as a compelling paradigm for systems that can reason, plan, and act toward complex goals. Prior work spans three complementary lines. (i) Tool-augmented single-agent methods enable an LLM to call external tools and interleave reasoning and acting, exemplified by ReAct ([27]) and Toolformer ([21]). (ii) Self-reflection and iterative improvement strategies, such as Reflexion ([22]), let agents monitor outcomes and refine plans or prompts to improve reliability. (iii) Multi-agent collaboration frameworks organize specialized roles (planners, executors, and critics) into coordinated protocols for complex tasks; representative systems include AutoGen [25], CodeAct [24], CAMEL [14], MetaGPT [12], and Voyager [23]. Guo et al. [11] survey LLM-based multi-agent systems, covering their application settings, agent profiling and communication, and capability acquisition. CHATDIFY follows the multi-agent line with specialized planner, executor, and critic roles that communicate through structured plans and diagnostic feedback.

The Proposed ChatDify Framework

CHATDIFY automates the synthesis of executable Dify workflows from natural language requirements. Formally, we define a Dify workflow WW as a triple W=(N,E,Θ)W = (N, E, \Theta), where N={n1,…,nk}N = \{n_1, \ldots, n_k\} denotes a set of heterogeneous functional nodes (e.g., LLM, Code), E⊆N×NE \subseteq N \times N represents directed data-flow dependencies forming a Directed Acyclic Graph (DAG), and Θ={θ1,…,θk}\Theta= \{\theta_1, \ldots, \theta_k\} specifies the dify-specific parameter configurations for each node.

As illustrated in Figure 2, CHATDIFY adopts an Actor-Critic architecture [17] (Konda and Tsitsiklis 1999) comprising three specialized agents: Planner, Executor, and Reviewer. The synthesis is modeled as a closed-loop optimization where agents collaboratively: (i) map user requests to a topological plan, (ii) perform context-aware synthesis and schema-augmented compilation into Dify’s Domain-Specific Language (DSL), and (iii) execute hybrid verification to drive closed-loop refinement.

Planner Agent: Hierarchical Decomposition and Pattern-Augmented Planning

Given a user’s request UU, the Planner Agent serves as the high-level architect, aiming to produce a structured execution plan P=(T,E)P = (T, E), where T={t1,t2,…,tk}T = \{t_1, t_2, \ldots, t_k\} is a set of sub-tasks and EE is a set of directed edges representing data dependencies between these tasks. This process is formulated as:

P=FPlanner(U,K)(1)P = \mathcal{F}_{\mathrm{Planner}}(U, K) \tag*{(1)}

where KK is the Dify Knowledge Base.

To ensure scalability and logical consistency, the Planner employs a top-down hierarchical decomposition and planning strategy:

  • Intent Recognition and Block Mapping: Upon receiving UU (e.g., “generate a competitive analysis report and email it”), the Planner parses the semantic intent to perform a functional decomposition, breaking the high-level goal into coarse-grained functional blocks (e.g., ‘Data Collection’ →\rightarrow ‘Information Processing & Analysis’ →\rightarrow ‘Report Generation’ →\rightarrow ‘Result Delivery’).

  • Pattern-Augmented Reference Retrieval: To avoid designing from scratch, the Planner performs semantic retrieval over KK to find the most relevant workflow sub-graph templates. This pattern-augmented approach ensures the generated plan inherits industry best practices (e.g., RAG pipelines or multi-hop reasoning chains).

  • DAG Synthesis and Node Selection: In the final planning stage, the Planner instantiates the abstract functional blocks into the concrete sequence of executable sub-tasks TT by leveraging retrieved templates and specific request constraints. For each sub-task, the Planner selects the optimal Dify functional nodes from KK (e.g., “Tavily Search”, “Knowledge Retrieval”, “LLM”). Simultaneously, it explicitly defines the data-flow dependencies EE to connect these nodes, thereby synthesizing the final topological DAG plan PP.

This plan PP acts as a precise “blueprint,” providing unambiguous logical specifications for the subsequent context-aware synthesis phase. As part of the closed-loop optimization process, the Planner is also responsible for re-planning and adjusting the plan PP based on high-level logical feedback from the Reviewer Agent.

Executor Agent: Context-Aware Synthesis and Schema-Augmented Compilation

The Executor Agent acts as the system’s “compiler,” tasked with synthesizing node parameters and compiling the abstract logical plan PP into a platform-specific, concrete, and executable Dify workflow WW. This process is defined as:

W=FExecutor(P,U,K)(2)W = \mathcal{F}_{\mathrm{Executor}}(P, U, K) \tag*{(2)}

where WW consists of instantiated nodes N={n1,…,nk}N = \{n_1, \ldots, n_k\}, their specific parameter configurations Θ={θ1,…,θk}\Theta= \{\theta_1, \ldots, \theta_k\}, and the connections between them (EE). KK provides the schema specifications for Dify nodes.

The key to the Executor’s operation lies in context-aware synthesis and schema-augmented compilation, which involves two core steps:

  • Context-Aware Parameter Synthesis: For each sub-task ti∈Tt_i \in T, the Executor instantiates the corresponding Dify node type from KK. The agent performs generative parameter synthesis to populate configuration θi\theta_i. This synthesis is conditioned not only on the user request UU and plan PP, but critically on the upstream execution context (e.g., outputs from predecessor nodes {n1,…,ni−1}\{n_1, \ldots, n_{i-1}\}). This allows for the dynamic construction of complex attributes, such as synthesizing task-specific system prompts for “LLM” nodes or executable logic for “Code” nodes. For underspecified parameters, a context-aware defaulting mechanism completes the configuration based on node functional semantics.

  • Schema-Augmented DSL Compilation: The Executor finally assembles all instantiated nodes and their edge connections into a unified YAML structure following the Dify DSL syntax. To enforce syntactic validity and reduce configuration errors, it applies a schema-augmented synthesis strategy. This mechanism restricts the generation process to strictly adhere to the multi-dimensional protocol specifications defined in KK, spanning key constraints such as: (i) Node-Specific Schemas, which dictate the mandatory and optional field structures for each node type, alongside their associated data types and range boundaries; (ii) Variable Propagation Protocols, validating the syntactic correctness and path-traceability of cross-node references (e.g., {{node_id.field}}).

The Executor Agent completes the critical synthesis and compilation from a logical task plan into an executable workflow instance on the Dify platform, effectively turning the “design blueprint” into “runnable code.” Within the closed-loop optimization process, the Executor receives specific error feedback from the Reviewer Agent regarding node parameters, configurations, or connections, and performs targeted repairs and re-instantiation.

Reviewer Agent: Hybrid Verification and Closed-loop Refinement

The Reviewer Agent serves as the Critic, validating WW through a hybrid process of static symbolic analysis and dynamic runtime verification:

F=FReviewer(W)=Fdynamic(Fstatic(W))(3)F = \mathcal{F}_{\mathrm{Reviewer}}(W) = \mathcal{F}_{\mathrm{dynamic}}\left(\mathcal{F}_{\mathrm{static}}(W)\right) \tag*{(3)}

where FF is a set of feedback signals. If F≠∅F \ne\emptyset, the framework enters an Iterative Repair Loop: W(t+1)=Refine⁡(W(t),F)W^{(t+1)} = \operatorname{Refine}(W^{(t)}, F), continuing until the workflow satisfies all constraints.

Static Symbolic Verification

The Reviewer performs a rigorous symbolic analysis without executing the workflow:

  • Topological Integrity: Ensures the graph is a valid DAG by detecting cycles, unreachable nodes, or broken data-flow edges, thereby guaranteeing a feasible execution order.

  • Type-Safe Variable Propagation: Constructs a variable dependency graph to trace all cross-node references (e.g., {{node_id.field}}). It verifies that every referenced variable is properly defined in an upstream node and that its data type matches the expected input schema of the consuming node.

  • Schema Compliance: Validates the YAML against Dify’s official schema, checking for missing required fields, parameter type mismatches, and out-of-range values.

Dynamic Runtime Probing

Static analysis cannot expose runtime-only defects, nor judge whether a successfully executed workflow actually fulfills the user’s intent. Once WW passes static checks, the Reviewer runs a two-stage probe in a controlled Dify sandbox:

  • Sandboxed Execution Probing: Autosynthesizes representative test inputs from UU and WW, executes WW end-to-end, and collects per-node status, traces, and outputs to surface hard execution failures (e.g., node exceptions, unresolved references, tool/API errors, missing credentials, timeouts).

  • LLM-as-Judge Semantic Verification: Since Dify workflows are open-ended (generation, summarization, tool use) and lack deterministic ground truth, an LLM judge consumes (U,W,trace,output)(U, W, \text{trace}, \text{output}) and returns a structured verdict with a node-level rationale, flagging cases where execution succeeds but behavior diverges from UU.

Experiments

Experimental Setup

Benchmark Dataset. We construct 100 production-inspired Dify workflow tasks covering eight categories, as shown in Table 1. These tasks are abstracted from common Dify application scenarios and workflow patterns.

Task CategoryCountPercentage
Text Generation Tasks1818%
Information Extraction1818%
Multimodal Understanding1111%
Multimodal Generation88%
Data Analysis and Processing1414%
Conversation and Interaction1010%
Code and Programming1010%
Research and Application1111%
Total100100%

Table 1. Task distribution across eight categories in the benchmark.

Models and Baselines. We instantiate ChatDify with Qwen-3.7 max [26], GLM-5.1 [28], and DeepSeek-v4 pro [4]. We compare with ZeroShot and ZeroShot+Dynamic, and ablate static verification, dynamic runtime probing, and their combination. For fair comparison, both ChatDify and ZeroShot+Dynamic run for up to three repair epochs by default.

Evaluation Protocol and Metrics. We report Success Rate (SR) and Token, the average token consumption in thousands. A task counts as successful if the workflow returned within the epoch budget passes three sequential checks: (i) a static check, requiring schema compliance, a valid DAG topology, and correctly defined variable references; (ii) a runtime check, where the workflow is imported into the Dify sandbox and executed end-to-end on a test case without node exceptions, unresolved references, tool/API errors, or timeouts; and (iii) a semantic check, where an LLM judge receives the request, the workflow DSL, the test case, the final outputs, and a per-node execution summary, and returns a binary verdict. The judge is the semantic verifier of the Reviewer Agent and uses the same backbone LLM as the generator. All methods, including ZeroShot and ZeroShot+Dynamic, are scored with this protocol.

Implementation Details. All experiments use Dify v1.11.2 and LLM APIs from Volcengine (GLM-5.1), Deepseek-AI (DeepSeek-v4 pro) and Alibaba Cloud (Qwen-3.7 max), running on a 4-core Intel(R) Xeon(R) Platinum CPU server with 16 GB RAM.

Main Results

Table 2 shows that direct prompting is insufficient for production-grade Dify synthesis. ZeroShot reaches 0.43 SR with Qwen-3.7 and only 0.13–0.15 with GLM-5.1 and DeepSeek-v4; adding dynamic repair improves SR to 0.67, 0.45, and 0.36, but remains far below the full system. CHATDIFY achieves 0.96, 0.91, and 0.75 SR across the three backends, improving over ZeroShot+Dynamic by 29, 46, and 39 percentage points respectively. This shows that reliability comes from the full Planner–Executor–Reviewer pipeline rather than runtime repair alone. Qwen-3.7 gives the strongest accuracy–cost profile (0.96 SR with 256k tokens), while DeepSeek-v4 is less effective on this benchmark. Across all backends, input tokens account for 80–90% of the reported totals, as retrieved knowledge base entries and the workflow DSL carried across refinement epochs accumulate in the prompt.

ModelMethodSRToken (k)
Qwen-3.7ZeroShot0.4324
ZeroShot+Dynamic0.6752
CHATDIFY (ours)0.96256
GLM-5.1ZeroShot0.1313
ZeroShot+Dynamic0.4536
CHATDIFY (ours)0.91348
DeepSeek-v4ZeroShot0.1515
ZeroShot+Dynamic0.3640
CHATDIFY (ours)0.75385

Table 2. Main comparison across LLM backends and practical baselines.

Table 3 shows that knowledge grounding alone is insufficient: it guides component selection but cannot verify whether a composed workflow executes correctly or fulfills the request. Removing dynamic probing therefore sharply reduces SR (0.96 to 0.73, 0.91 to 0.71, and 0.75 to 0.44), while static verification adds a complementary 3–8 percentage point gain by filtering structural errors before execution. The lowest SR without both stages (0.60 and 0.33 for GLM-5.1 and DeepSeek-v4) validates their complementary roles in reliable synthesis.

ModelMethodSRToken (k)
Qwen-3.7Full CHATDIFY0.96256
w/o Static0.93270
w/o Dynamic0.73175
w/o Static & Dynamic0.71166
GLM-5.1Full CHATDIFY0.91348
w/o Static0.83426
w/o Dynamic0.71219
w/o Static & Dynamic0.60210
DeepSeek-v4Full CHATDIFY0.75385
w/o Static0.67397
w/o Dynamic0.44162
w/o Static & Dynamic0.33145

Table 3. Ablation results on static verification and dynamic runtime probing.

Finally, Table 4 shows the cost–quality trade-off of refinement. Most gains appear within two epochs (Qwen-3.7: 0.73 to 0.95; GLM-5.1: 0.71 to 0.88; DeepSeek-v4: 0.44 to 0.64), while the third epoch brings smaller gains at higher token cost. Thus, two epochs suit high-throughput deployment, whereas three epochs favor maximum success rate.

ModelEpochsSRToken (k)
Qwen-3.710.73175
20.95244
30.96256
GLM-5.110.71219
20.88305
30.91348
DeepSeek-v410.44162
20.64288
30.75385

Table 4. Effect of iterative refinement epochs on workflow synthesis performance. Increasing epochs enables additional closed-loop repair but incurs higher token consumption.

Conclusions

We introduced CHATDIFY, a multi-agent framework for synthesizing Dify workflows from natural language. Its Planner–Executor–Reviewer architecture bridges user intent and platform-specific DSL configuration through knowledge-grounded planning, schema-aware compilation, and hybrid verification. On 100 production-inspired tasks, CHATDIFY consistently outperforms practical zero-shot baselines, validating its value for industrial Dify workflow automation. Although our implementation targets Dify, the core design is extensible to other workflow orchestration platforms that expose structured node schemas, executable DSLs, and runtime validation interfaces.

Limitations

ChatDIFY is implemented and evaluated on Dify, which allows us to study NL-to-workflow synthesis under a concrete industrial DSL with real deployment constraints. This focus also means that direct transfer to other workflow platforms requires adapting the knowledge base, node schemas, and sandbox executor to the target platform. However, these components are modular in our Planner–Executor–Reviewer design, and the same principles apply to platforms that expose structured node specifications and executable workflow definitions.

Our benchmark contains 100 production-inspired tasks, covering major Dify workflow categories and common application patterns. While this scale is sufficient to compare design choices and baselines, broader cross-organization benchmarks would further improve external validity. In addition, dynamic probing and LLM-as-Judge verification introduce extra token cost, reflecting an engineering trade-off between reliability and inference budget. Our epoch analysis suggests that this cost can be tuned in practice: two refinement rounds provide strong performance for high-throughput settings, while additional rounds can be reserved for higher-stakes workflows.

Finally, SR measures execution success together with LLM-judged semantic success, and does not fully reflect human judgment of task completion. We adopt this automated protocol because open-ended workflow outputs, such as generated reports or replies, lack deterministic ground truth, and our experiments produce thousands of workflows across backbones, runs, baselines, and ablations, which makes exhaustive human evaluation impractical. Reusing the Reviewer’s verifier as the judge also keeps the evaluation aligned with the checks that drive refinement. The cost of this choice is that each check relies on a single synthesized test case and the judge shares its backbone LLM with the generator, so SR may overestimate the true success rate. Although our manual spot checks found that most workflows behaved as expected, a systematic human evaluation with multiple test cases and independent annotators would provide a stricter assessment.

Dify Workflow Example

This section presents an example of complex Dify workflows to illustrate the structure and complexity of real-world workflow automation on the Dify platform. We use Dify DeepResearch as a representative example, which demonstrates how sophisticated research tasks can be orchestrated through a visual workflow interface. The Figure 3 shows the workflow interface and its corresponding YAML configuration.

Workflow interface demonstrating complex research task automation

(a) Workflow interface demonstrating complex research task automation

Partial YAML configuration

(b) Partial YAML configuration

Figure 3. Example of Dify DeepResearch workflow (from dify.ai): (a) workflow interface and (b) partial YAML configuration.

Dify Knowledge Base

The Dify Knowledge Base serves as a comprehensive repository that provides structured specifications and exemplars for all available Dify nodes and their interconnections. The knowledge base is organized into four complementary components, each serving a distinct purpose in the workflow synthesis process.

Edge Usage Specifications

The Edge Usage Specifications component defines how nodes can be connected within workflows. For each node type, it provides:

  • Usage Specifications: A description of the node’s role and capabilities within workflow applications, including its primary functions and supported operation modes.

  • Application Scenarios: A structured categorization of connection patterns:

  • Applicable Scenarios: Valid connection patterns that represent common and recommended usage patterns. For example, LLM nodes can connect to knowledge retrieval nodes for RAG systems, parameter extraction nodes for structured data processing, conditional branch nodes for decision-making, and various other node types depending on the task requirements.

  • Inapplicable Scenarios: Invalid or problematic connection patterns that should be avoided. These include direct self-loops, incomplete workflow structures (e.g., LLM nodes that are the only endpoint without connecting to output nodes), and other patterns that violate Dify’s workflow constraints.

This component guides the Planner Agent in constructing valid execution plans and helps the Reviewer Agent detect structural issues during static verification. Figure 4 presents an example of edge usage specifications for LLM nodes, detailing their connection patterns and constraints.

Edge Usage Specifications for LLM node type

Figure 4. Example of Edge Usage Specifications for LLM node type

Edge Examples

The Edge Examples component contains validated edge configurations that demonstrate correct node-to-node connections. These examples illustrate proper data flow patterns and help ensure that the generated workflows maintain correct topological ordering and variable scoping.

As an illustration, Figure 5 demonstrates valid edge configurations, showing connections from a start node to an LLM node and from an LLM node to a tool node, including source and target node types, edge metadata, connection handles, and contextual information.

Example of Edge Configurations showing valid node connections

Figure 5. Example of Edge Configurations showing valid node connections

Node Usage Specifications

The Node Usage Specifications component provides detailed guidelines for configuring each node type. For each node type, it includes:

  • Input Specifications: A comprehensive list of required and optional input parameters, their data types, constraints, and configuration requirements. For instance, for LLM nodes, this includes specifications for prompt templates (supporting both Chat and Completion modes), model configuration (provider, model name, and parameters), variable selectors, Jinja2 variables, vision and context configurations, and conversation history settings.

  • Common Usage Errors: A curated list of frequently encountered configuration mistakes and validation rules. These error patterns help the Reviewer Agent identify potential issues during static verification. Examples include ensuring non-empty prompt templates, validating variable references against the variable pool, verifying correct variable path formats, and checking that referenced nodes are valid predecessors in the workflow graph.

This component enables the Executor Agent to make informed decisions about parameter initialization and helps prevent common configuration errors during workflow synthesis. Figure 6 illustrates a concrete example of node usage specifications for an LLM node type.

Example of Node Usage Specifications for LLM node type

Figure 6. Example of Node Usage Specifications for LLM node type

Node Examples

The Node Examples component contains concrete, validated node configurations in Dify’s DSL format. Each example demonstrates a complete node instantiation with all required fields populated according to the node type. These examples serve as templates that the Executor Agent can adapt based on the user’s requirements and the structured plan from the Planner Agent. By retrieving semantically similar examples from this component, the Executor can initialize node parameters more accurately, reducing the need for iterative refinement.

As an illustration, Figure 7 shows a complete node configuration example for an LLM node used within an iteration block, including model configuration, prompt templates, variable bindings, and structural properties.

Example of Node Configuration for an LLM node within an iteration block

Figure 7. Example of Node Configuration for an LLM node within an iteration block

Knowledge Base Integration

The four components work together to support the multi-agent workflow synthesis process. During planning, the Planner Agent queries the Edge Usage Specifications and Edge Samples to construct valid execution plans. During execution, the Executor Agent retrieves relevant Node Examples and adapts them based on Node Usage Specifications to instantiate configured nodes. The Reviewer Agent leverages all four components to perform comprehensive validation: it uses Node Usage Specifications to check parameter correctness, Edge Usage Specifications to verify structural validity, and compares the generated workflow against the examples to identify deviations from established patterns.

This structured knowledge representation enables CHATDIFY to generate workflows that are not only syntactically correct but also follow Dify’s best practices and common usage patterns, significantly improving the success rate and quality of generated workflows.

Case Study

We present several representative case studies that demonstrate CHATDIFY’s capability to handle diverse workflow requirements.

Competitive Analysis Report

User Request: "Generate an analysis and a competitive report for three mainstream AI Agent tools, namely Dify, Coze and N8n. The report shall cover the following comparison dimensions: core positioning, key technical features, learning curve, deployment and data security, pricing model, strengths and weaknesses, typical applicable scenarios, etc. The output format of the research report shall be a Markdown document."

Synthesized Dify Workflow: CHATDIFY synthesizes a sequential workflow: (i) a start node that receives user inputs including analysis requirements, report depth, comparison dimensions, and target audience, (ii) an LLM node for research strategy planning, (iii) a Tavily search tool node for information collection, (iv) an LLM node for competitive analysis, (v) a Markdown converter tool node for report formatting, and (vi) an end node that outputs the final competitive analysis report. The workflow is shown in Figure 8.

CHATDIFY-synthesized dify workflow for competitive analysis report generation

Figure 8. CHATDIFY-synthesized dify workflow for competitive analysis report generation.

Public Opinion Report with Iteration

User Request: "Generate a public opinion report based on the latest news relevant to the user’s keywords. For each news article, extract its title and conduct sentiment analysis, then compile the findings into a comprehensive report."

Synthesized Dify Workflow: CHATDIFY synthesizes a workflow with iterative processing: (i) a start node that receives user inputs including keywords, time range, and maximum number of news articles, (ii) a Tavily search tool node for retrieving relevant news articles, (iii) a code node that extracts and validates news URLs from the search results, (iv) an iteration node that processes each news URL

in parallel, containing a Jina Reader tool node for content extraction and an LLM node for sentiment analysis, (v) an LLM node that aggregates all sentiment analysis results to generate a comprehensive public opinion report, and (vi) an end node that outputs the final report. The workflow is shown in Figure 9.

CHATDIFY-synthesized dify workflow for public opinion report generation

Figure 9. CHATDIFY-synthesized dify workflow for public opinion report generation.

Intelligent Contract Analysis

User Request: "Generate an intelligent contract analysis workflow that supports batch processing of contract documents, automatically identifies risks, and generates analysis reports. The workflow should iteratively process all uploaded contract documents. For each document, it should: (i) identify and verify whether it is a contract-type file (non-contract files should be directly prompted and the process terminated), (ii) automatically classify contract types (procurement, labor, lease, service, etc.), (iii) extract key information including contract subject information (contracting parties, contact information, qualifications), core clauses (rights and obligations, payment methods, delivery standards, acceptance conditions), critical dates (signing date, effective date, expiration date, payment deadlines, delivery deadlines), and financial information (total contract amount, payment ratios, penalty amounts), (iv) perform risk detection and rating including clause completeness checks, potential risk identification, compliance checks, critical date reminders, and risk level assessment (high/medium/low risk). Finally, generate an overall analysis report including contract quantity, contract type distribution, risk level distribution, and risk point distribution in Markdown document format."

Synthesized Dify Workflow: CHATDIFY synthesizes a workflow with iterative batch processing: (i) a start node that receives user inputs including contract documents, analysis scope, and additional analysis requirements, (ii) an iteration node that processes each contract document in parallel, containing: (a) a document extractor node for content extraction, (b) an LLM node for contract identification analysis, (c) a code node that parses the identification results and converts them to boolean judgment conditions, (d) an if-else node that branches based on contract identification results, (e) for contract files: an LLM node for deep contract analysis (classification, key information extraction, and preliminary risk identification), a knowledge retrieval node for retrieving relevant laws, regulations, and risk detection rules, an LLM node for comprehensive risk assessment and rating, and a variable aggregator node for collecting analysis results, (f) for non-contract files: an answer node that outputs a prompt message, (iii) an LLM node that generates an overall analysis report based on aggregated contract analysis data, (iv) a Markdown formatting tool node that converts the report to standard Markdown format, and (v) an end node that outputs the final intelligent contract analysis report. The workflow is shown in Figure 10.

CHATDIFY-synthesized dify workflow for intelligent contract analysis

Figure 10. CHATDIFY-synthesized dify workflow for intelligent contract analysis.

References

  1. [1]Eleni Adamopoulou and Lefteris Moussiades. 2020. An overview of chatbot technology. In IFIP international conference on artificial intelligence applications and innovations, pages 373–383. Springer.
  2. [2]Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noven Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261.arxiv.org/abs/2507.06261
  3. [3]Coze. 2024. Coze: Connect ai to your apps.
  4. [4]DeepSeek-AI. 2026. Deepseek-v4: Towards highly efficient million-token context intelligence.arxiv.org/abs/2606.19348
  5. [5]Dify AI. 2023. Dify: Build production-ready agentic workflow.
  6. [6]Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783.arxiv.org/abs/2407.21783
  7. [7]Akiharu Esashi, Pawissanutt Lertpongruj ikorn, Mohsen Amini Salehi, and Shinji Kato. 2025. Action engine: Automated workflow generation in faas. Future Generation Computer Systems, page 107947.arxiv.org/abs/2411.19485
  8. [8]Shengda Fan, Xin Cong, Yuepeng Fu, Zhong Zhang, Shuyan Zhang, Yuanwei Liu, Yesai Wu, Yankai Lin, Zhiyuan Liu, and Maosong Sun. 2024. Workflowllm: Enhancing workflow orchestration capability of large language models. arXiv preprint arXiv:2411.05451.arxiv.org/abs/2411.05451
  9. [9]Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Meng Wang, and Haofen Wang. 2023. Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997.arxiv.org/abs/2312.10997
  10. [10]Thomas Gschwind, Shramona Chakraborty, Nitin Gupta, and Sameep Mehta. 2025. Classifier-augmented generation for structured workflow prediction. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 1227–1238.arxiv.org/abs/2510.12825
  11. [11]Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V. Chawla, Olaf Wiest, and Xiangliang Zhang. 2024. Large language model based multi-agents: A survey of progress and challenges. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI-24, pages 8048–8057. International Joint Conferences on Artificial Intelligence Organization. Survey Track.
  12. [12]Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, et al. 2023. Metagpt: Meta programming for a multi-agent collaborative framework. In The Twelfth International Conference on Learning Representations. Vijay Konda and John Tsitsiklis. 1999. Actor-critic algorithms. Advances in neural information processing systems, 12.
  13. [13]Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems, 33:9459–9474.arxiv.org/abs/2005.11401
  14. [14]Guohao Li, Hasan Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. 2023. Camel: Communicative agents for "mind" exploration of large language model society. Advances in Neural Information Processing Systems, 36:51991–52008.arxiv.org/abs/2303.17760
  15. [15]Zelong Li, Shuyuan Xu, Kai Mei, Wen yue Hua, Balaji Rama, Om Raheja, Hao Wang, He Zhu, and Yongfeng Zhang. 2024. Autoflow: Automated workflow generation for large language model agents. arXiv preprint arXiv:2407.12821.arxiv.org/abs/2407.12821
  16. [16]Hanchao Liu, Rongjun Li, Weimin Xiong, Ziyu Zhou, and Wei Peng. 2025. Workteam: Construction workflows from natural language with multi-agents. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 3: Industry Track), pages 20–35.
  17. [17]Ryan Lowe, Yi I Wu, Aviv Tamar, Jean Harb, OpenAI Pieter Abbeel, and Igor Mordatch. 2017. Multi-agent actor-critic for mixed cooperative-competitive environments. Advances in neural information processing systems, 30.arxiv.org/abs/1706.02275
  18. [18]Vasilios Mavroudis. 2024. Langchain.DOI
  19. [19]OpenAI. 2022. Chatgpt.
  20. [20]OpenAI. 2023. Gpt-4 technical report. ArXiv, abs/2303.08774.arxiv.org/abs/2303.08774
  21. [21]Timo Schick, Jane Dwivedi-Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems, 36:68539–68551.arxiv.org/abs/2302.04761
  22. [22]Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36:8634–8652.arxiv.org/abs/2303.11366
  23. [23]Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. 2023. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291.arxiv.org/abs/2305.16291
  24. [24]Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, and Heng Ji. 2024. Executable code actions elicit better llm agents. In Forty-first International Conference on Machine Learning.arxiv.org/abs/2402.01030
  25. [25]Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, et al. 2024. Autogen: Enabling next-gen llm applications via multi-agent conversations. In First Conference on Language Modeling.arxiv.org/abs/2308.08155
  26. [26]An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Cheng en Huang, Chenxu Lv, et al. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388.arxiv.org/abs/2505.09388
  27. [27]Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. In The eleventh international conference on learning representations.arxiv.org/abs/2210.03629
  28. [28]Aohan Zeng, Xin Lv, Zhenyu Hou, Zhengxiao Du, Qinkai Zheng, Bin Chen, Da Yin, Chendi Ge, Chenghua Huang, Chengxing Xie, et al. 2026. Glm-5: from vibe coding to agentic engineering. arXiv preprint arXiv:2602.15763.arxiv.org/abs/2602.15763

Paper details

vlnc:W_5ajfeqph
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents