betaFeedback

WorksComputer Science

CutAgent: Creative-Gene-Driven Planning and Experience-Guided Self-Refinement for High-CTR E-commerce Advertising Video Creation

Abstract

E-commerce advertising video creation requires both high-level creative planning and reliable multi-modal production. However, existing methods often map product attributes directly to scripts or editing decisions without explicitly modeling advertising strategies, narrative structures, and production styles. They also have limited ability to reuse past production failures. To address these limitations, we propose CutAgent\textbf{CutAgent}, a dual-agent framework that formulates advertising video creation as a Plan–Execute–Self-Evolve process. CutAgent comprises two collaborative agents, GeneAgent\textbf{GeneAgent} and VideoAgent\textbf{VideoAgent}, connected by a structured creative gene that bridges product understanding and video production. GeneAgent maps multi-modal product inputs—textual attributes together with product images—to creative genes that specify narrative units and production rules. To train GeneAgent, we construct CutAgent-70K\textbf{CutAgent-70K} with approximately 70K samples of product attributes/images, reverse-extracted creative genes, cleaned clip pools, and real-world CTR, and apply high-CTR supervised fine-tuning followed by CTR-guided GRPO. Guided by the generated gene, VideoAgent retrieves category-then-gene failure–correction experiences from a long-term memory as constraints for shot-script planning, then coordinates material retrieval, narration, subtitles, music, and timeline rendering. Independently, a reflection loop diagnoses quality defects, revises deficient production decisions, and writes verified corrections back to memory for later retrieval. Experiments on CutAgent-Eval\textbf{CutAgent-Eval} show that CutAgent outperforms representative baselines in e-commerce advertising video creation, achieving the highest online CTR of 2.21%\textbf{2.21\%}.

Introduction

On modern e-commerce platforms, short-form advertising videos have become an important medium for product promotion and user engagement [21, 28, 46]. Compared with static product pages that primarily present specifications and prices, short videos can demonstrate product usage, communicate selling points, and encourage immediate user interaction within a limited duration. In this setting, click-through rate (CTR) provides a direct behavioral signal of whether an advertisement successfully captures attention, conveys product value, and motivates users to engage. Consequently, platforms and merchants increasingly demand scalable methods for producing advertising videos with strong conversion potential.

Producing such videos, however, is far from a single generation step. An effective advertisement must decide which selling points to emphasize, in what narrative order they should appear, and how they should be realized through materials, scripts, voice-over, subtitles, and background music. Professional editors routinely iterate over these intertwined decisions, yet the cost of manual production grows rapidly with the number of SKUs, seasonal campaigns, and category variations. This tension between conversion-oriented creative quality and production scalability motivates automated high-CTR advertising video creation.

While automated video production has rapidly advanced from template-based retrieval [3] to LLM-guided planning [17, 25] and sophisticated agentic workflows [10, 45, 46], generating high-CTR advertising videos remains under-addressed along two critical dimensions. First, existing pipelines map product attributes directly to low-level editing decisions without a globally coherent advertising strategy. Because creative intent is entangled with execution, it becomes exceedingly difficult to optimize the planning process using commercial feedback like CTR. Second, advertising video production is inherently error-prone. Current systems rarely diagnose failures—such as missing selling points or mismatched audio—at the decision level, nor do they reuse past failure experiences to prevent the same mistakes on subsequent products.

Figure 1. Illustration of CutAgent on a tea-separation glass bottle advertisement. Given product information and assets, CutAgent generates a creative-gene-guided shot sequence covering attention capture, product recognition, feature demonstration, trust building, and purchase guidance. The reflection loop diagnoses weak feature emphasis in the initial draft and refines the video with stronger demonstrations, daily-use scenarios, and improved audio rhythm.

To close this gap, we propose CutAgent, a dual-agent framework that formulates e-commerce advertising video creation as a Plan–Execute–Self-Evolve paradigm. By introducing an explicit creative gene (i.e., ordered narrative units and production rules) as an intermediate representation, CutAgent decouples conversion-oriented creative planning from low-level production operations. This design ensures that the generated videos adhere to a globally coherent advertising strategy while making the pipeline explicitly controllable, amenable to CTR-guided learning, and capable of continuous self-refinement.

Specifically, CutAgent operates through two synergistic agents to map multi-modal product inputs to high-CTR video outputs. In the planning stage, the GeneAgent processes product attributes to generate the creative gene. To ensure commercial effectiveness, it is trained on our newly constructed CutAgent-70K dataset using supervised fine-tuning [27] on top-tier data, followed by CTR-guided Group Relative Policy Optimization (GRPO) [30] with a surrogate reward model [15]. Subsequently, the VideoAgent handles execution and self-evolution. Guided by the generated gene and a long-term memory of past failure–correction experiences, it coordinates script expansion and video rendering. Crucially, VideoAgent employs an independent post-production reflection loop to diagnose deficient decisions, revise affected components, and write verified corrections

back into memory for future tasks. Extensive experiments on our CutAgent-Eval benchmark demonstrate that CutAgent successfully automates and optimizes ad generation, outperforming representative methods and achieving a highly competitive online CTR of 2.21%.

Our main contributions are summarized as follows:

  • We construct CutAgent-70K, a large-scale dataset connecting multi-modal product attributes and images, reverse-extracted creative genes, cleaned video-clip pools, and real-world CTR signals, together with CutAgent-Eval for comprehensive evaluation of advertising video planning and production.

  • We introduce GeneAgent, a conversion-oriented creative planning agent that learns product-conditioned creative genes from multi-modal inputs of product attributes and product images through high-CTR supervised fine-tuning and CTR-guided GRPO.

  • We propose VideoAgent, which translates creative genes into complete advertising videos under category-then-gene constraints retrieved from a long-term memory, and independently performs six-dimensional diagnosis with reflection-based iterative refinement, writing verified corrections back to memory.

  • Extensive experiments demonstrate that the resulting CutAgent framework consistently outperforms representative open-source and proprietary methods on CutAgent-Eval.

Related Work

E-commerce Video Creation

Recent studies have explored agentic frameworks for automating complex video editing workflows [36, 42]. Cut-Claw [45] employs hierarchical audiovisual decomposition together with Playwriter, Editor, and Reviewer agents to transform hours-long raw footage into instruction-following, music-synchronized short videos. FireRed-OpenStoryline [10] adopts a tool-oriented architecture in which language models coordinate media retrieval, clip understanding, script generation, timeline planning, and rendering.

For e-commerce video creation, AutoCut [46] discretizes visual and audio representations into multi-modal tokens and fine-tunes a multi-modal language model for clip selection, clip ordering, script generation, and background music selection. Although AutoCut uses click-through rate and like rate to select high-engagement training videos, these signals are used for data curation rather than as explicit optimization objectives. Existing methods [10, 45, 46] therefore lack a unified mechanism that combines conversion-oriented planning, post-production reflection, and reusable correction experience across production tasks. CutAgent addresses this gap by separating creative-gene planning from agentic video production and accumulating verified failure–correction experiences in a category-then-gene long-term memory.

Planning and Self-Evolving Agents

Large language model (LLM) agents have emerged as a general paradigm for solving complex tasks through planning, tool use, and interaction with external environments [38]. Among representative approaches, ReAct [41] interleaves reasoning with environmental actions, allowing an agent to update its decisions based on intermediate observations, while HuggingGPT [31] uses a language model for task planning, model selection, subtask execution, and result aggregation.

Beyond single-task execution, self-evolving agents seek to improve their future behavior by acquiring, refining, and reusing experiences from previous interactions [11]. Among representative approaches, Reflexion [32] transforms evaluation feedback into verbal reflections stored in episodic memory, whereas ExpeL [44] extracts reusable insights and successful experiences from historical trajectories. However, these approaches mainly target general reasoning and tool-use tasks, without jointly considering structured creative planning, multi-modal production, outcome-driven evaluation, and the reuse of dimension-specific production experiences. CutAgent extends this paradigm to e-commerce advertising video creation by separating creative-gene planning from video execution and accumulating verified failure–correction experiences in a category-then-gene long-term memory.

Method

Overview

Figure 2 illustrates the overall CutAgent pipeline. The input of CutAgent is a multi-modal product representation PiP_i that contains both textual product attributes. Conditioned on PiP_i, GeneAgent performs conversion-oriented planning and outputs a structured creative gene G~i\widetilde{G}_i. As illustrated in the figure, the gene comprises an ordered sequence of narrative units and corresponding audiovisual production rules. This gene answers what kind of advertisement should be created before any clip is selected.

Figure 2. Overview of CutAgent. GeneAgent takes product attributes and product images as multi-modal input PiP_i and maps them to a creative gene (narrative units with voice-over, music and script rules). Conditioned on the gene, VideoAgent first retrieves category-then-gene failure–correction experiences from a long-term memory E\mathcal{E}, then executes creative-gene-guided shot planning, video-clip retrieval, TTS and subtitle generation, BGM matching, and timeline rendering. Independently, a reflection loop performs evaluation, diagnosis, revision, and verification after rendering, and writes verified experiences back to E\mathcal{E} for later retrieval.

Conditioned on the product and creative gene, VideoAgent executes the video production. Crucially, as illustrated in Figure 2, this process is augmented by two independent modules: a memory-read step before script expansion, and a reflection-write step after rendering. First, given a new gene G~i\widetilde{G}_i, VideoAgent retrieves failure–correction experiences from long-term memory. Second, the agent executes the script by retrieving visual clips from a product-specific pool Mi\mathcal{M}_i, synthesizing audio and subtitles and rendering the final timeline. Finally, an independent reflection loop evaluates the rendered video, diagnoses deficient decisions, and writes newly verified failure–correction pairs back into E\mathcal{E} to prevent recurring mistakes in future tasks.

CutAgent therefore follows Plan–Execute–Self-Evolve: GeneAgent plans the creative gene, VideoAgent executes multi-modal production under memory-retrieved constraints, and reflection enables non-parametric self-evolution by updating E\mathcal{E} across products.

GeneAgent

Creative Gene Representation

For the ii-th product, the multi-modal input is

Pi={ci,ti,pi,ai,Iimain,Iidetail,ui},(1)P_i = \{c_i, t_i, p_i, a_i, I_i^{\mathrm{main}}, I_i^{\mathrm{detail}}, u_i\}, \tag*{(1)}

where cic_i, tit_i, pip_i, and aia_i denote textual product attributes (category, SKU title, price, and selling points), IidetailI_i^{\mathrm{detail}} and IidetailI_i^{\mathrm{detail}} are the main image and detail images of the product, and uiu_i is the review summary. Thus PiP_i always couples product attributes with product images, which is necessary for generating shot plans that remain visually aligned with the target product.

GeneAgent is a vision–language model that consumes the complete multi-modal product representation PiP_i. Following the creative-gene prompt in Figure 6(A), it predicts a structured creative gene:

Prompt templates used in CutAgent

Figure 6. Prompt templates used in CutAgent. (A) Creative-gene generation prompts used by GeneAgent at training and inference. (B) VideoAgent prompts for shot-script planning (B1) and narration/subtitle adaptation (B2). (C) Reverse-extraction prompts that recover creative genes from historical advertising videos for CutAgent-70K supervision: voice attribute analysis (C1), BGM style analysis (C2), shot-level multimodal analysis (C3), and creative-gene reconstruction (C4).

G~i=GeneAgent⁡(Pi)=(S~i,v~i,m~i,r~s,i),(2)\widetilde{G}_i = \operatorname{GeneAgent}(P_i) = (\widetilde{S}_i,\widetilde{v}_i,\widetilde{m}_i,\widetilde{r}_{s,i}), \tag*{(2)}

where

S~i=[s~i,1,…,s~i,ni], s~i,j=(u~i,j,d~i,j), u~i,j∈NU.(3)\widetilde{S}_i = [\widetilde{s}_{i,1}, \ldots, \widetilde{s}_{i,n_i}],\ \widetilde{s}_{i,j} = \left(\widetilde{u}_{i,j}, \widetilde{d}_{i,j}\right),\ \widetilde{u}_{i,j} \in\mathcal{N}U. \tag*{(3)}

Here, S~i\widetilde{S}_i is an ordered sequence of narrative units. Each narrative unit consists of a discrete type u~i,j\widetilde{u}_{i,j} from the predefined vocabulary NU\mathcal{N}U and a product-conditioned textual realization d~i,j\widetilde{d}_{i,j} specifying how the corresponding narrative function should be instantiated for the target product (the complete 27-unit vocabulary is listed in Table 3).

CodeNarrative UnitDescriptionTypical Scene
Attention Capture
NU-L1Price ShockHighlights an unexpectedly attractive price to create immediate surprise.Large price caption, such as “Only $9.9.”
NU-B3Trend-Based OpeningIntroduces the product through a current trend, popular topic, or Internet meme.Trending scenarios or popular online expressions.
NU-P0Visual ImpactCaptures attention through striking product effects, transformations, or contrasts.Dynamic product presentation or before–after comparison.
NU-O4Suspense PreviewPresents the final result first and then reveals how it is achieved.An impressive result followed by a suspenseful question.
NU-D3Pain-Point ScenarioPresents the inconvenience experienced before using the product.A realistic scene in which the user encounters a problem.
Product Introduction
NU-Q7Full Product ViewPresents the complete appearance and overall design of the product.Front, panoramic, or multi-angle product views.
NU-C1Detail Close-UpHighlights the material, craftsmanship, texture, or local product details.Macro or close-up shots of product details.
NU-T8UnboxingShows the unpacking and initial product-reveal process.Opening the package and removing the product.
NU-D7Package & AccessoriesPresents the packaging, included accessories, and complimentary items.Flat-lay display of the package and accessories.
NU-G8SpecificationsDisplays product parameters, ingredients, dimensions, or specifications.Close-up of a specification label or ingredient list.
NU-U9Product VariantsPresents multiple colors, styles, sizes, or product variants.Multiple SKUs displayed side by side or sequentially.
Functional Demonstration
NU-H9Core Function DemoDemonstrates one primary function of the product.Operating the product to show its core function.
NU-N5Usage TutorialExplains how to use the product through sequential operating steps.Step-by-step product-use demonstration.
NU-R9Effect DemonstrationPresents the immediate outcome or effect produced by the product.Before–after comparison or direct result display.
NU-X2Usage ScenarioDemonstrates product use in a realistic application scenario.Using the product in a kitchen, bedroom, or outdoor environment.
NU-Z2Multi-Scenario DemoShows that the product can be applied across multiple usage scenarios.Switching between different product-use environments.
NU-S6A/B ComparisonCompares the product with a competing product or conventional solution.Split-screen or sequential comparison.
Trust Building
NU-A7Influencer EndorsementBuilds credibility through a recommendation from an influencer or expert.An influencer appears on screen and recommends the product.
NU-V4User TestimonialUses customer feedback or testimonials to establish product credibility.Customer-review screenshots or on-camera testimonials.
NU-I6Authoritative CertificationPresents certificates, test reports, or authoritative endorsements.Close-up of a certificate or inspection report.
NU-F5Technology ExplanationExplains the product’s ingredients, mechanism, or underlying technology.Ingredient lists accompanied by explanatory graphics.
NU-Y3Brand Story & ProvenanceIntroduces the brand history, manufacturing process, or product origin.Factory, place-of-origin, or brand-history footage.
Promotional Conversion
NU-W6Price/Discount RevealReveals the product price, discount, or promotional offer.Animated price label or spoken price announcement.
NU-J4Limited-Time PromotionCreates urgency through a limited-time or limited-stock offer.Countdown timer or limited-stock label.
NU-K3Gift or Bundle ValueHighlights complimentary items, bundles, or package value.Display of free gifts or a complete product bundle.
NU-M2Purchase GuidanceExplicitly directs the viewer toward the purchase action.Arrow animation accompanied by “Click the link.”
Closing
NU-E0Branded End CardConcludes the video with brand identity and a consistent visual ending.Brand logo animation accompanied by a slogan.

Table 3. Complete narrative-unit registry used by CutAgent.

The remaining fields denote the voice-over, music and script rules, respectively. The creative gene serves as an explicit interface between CTR-oriented creative planning and downstream video production.

Dataset Construction

We construct CutAgent-70K for GeneAgent training and VideoAgent material retrieval, and independently curate CutAgent-Eval for final evaluation (see Figure 3(a)).

Figure 3. Construction of CutAgent-70K and CTR-guided GeneAgent training. (a) For each product, historical ads are cleaned into a clip pool and processed by reverse gene extraction for CutAgent-70K (see Figure 6(C1–C4)), followed by quality control; the resulting sample contains product attributes and images PiP_i, gene GiG_i, cleaned clips MiM_i, and CTR yiy_i. (b) GeneAgent (Qwen3-VL-8B) is first supervised on high-CTR samples, then optimized by GRPO: K=8K=8 candidate genes are sampled and scored by a composite reward (CTR effectiveness 0.60, format compliance 0.25, gene diversity 0.15) using a frozen LightGBM CTR-level predictor as the training-time surrogate reward (not for evaluation).

CutAgent-70K contains approximately 70K product-level samples covering 43 product categories organized into 8 portfolios (see Figure 4). For each product, we collect multi-modal product information PiP_i, including textual attributes and main/detail images, together with a product-specific material pool Mi\mathcal{M}_i containing approximately 90–120 cleaned video clips. We further recover the corresponding creative gene GiG_i through a four-stage reverse-extraction pipeline (see Figure 6C). The resulting dataset is

Category distribution of CutAgent-70K across 43 categories and 8 portfolios

Figure 4. Category distribution of CutAgent-70K across 43 categories and 8 portfolios. Outer bars indicate relative category mass; inner labels report portfolio-level mean share.

D70K={(Pi,Gi,Mi,yi)}i=1N,N≈70K,(4)\mathcal{D}_{70K} = \left\{(P_i, G_i, \mathcal{M}_i, y_i)\right\}_{i=1}^{N},\quad N \approx70\mathrm{K}, \tag*{(4)}

where yi=Ci/Eiy_i = C_i/E_i is the observed CTR. GeneAgent training uses (Pi,Gi,yi)(P_i, G_i, y_i); VideoAgent retrieves shots from Mi\mathcal{M}_i at production time.

Cleaning uses ffmpeg scene detection, removes clips with on-camera humans or under 2 s, blurs subtitles/watermarks, and deduplicates by perceptual hash. Reverse extraction uses Qwen3.5-Omni-Plus-Realtime for voice/BGM attributes and Doubao-Seed-2.0-pro [29] for shot analysis and gene reconstruction.

CutAgent-Eval

CutAgent-Eval is an independent 5K-product set from the same 43-category taxonomy, unused in SFT, reward-model training, or GRPO; all methods share its product inputs and standardized clip pools.

GeneAgent Optimization

As illustrated in Figure 3(b), GeneAgent is optimized in two stages. High-CTR supervised fine-tuning first establishes the product-to-gene mapping, after which CTR-guided GRPO improves the advertising potential of generated creative genes.

High-CTR Supervised Fine-Tuning

We select the top 20% samples ranked by observed CTR from CutAgent-70K as high-quality creative demonstrations:

DSFT=Top⁡20%(D70K;CTR).(5)\mathcal{D}_{\mathrm{SFT}} = \operatorname{Top}_{20\%}(\mathcal{D}_{70K};\mathrm{CTR}). \tag*{(5)}

We employ Qwen3-VL-8B-Instruct [2] as the vision–language backbone of GeneAgent. For each training sample, the model input is the full multi-modal product representation PiP_i, i.e., textual product attributes together with the main and detail product images, under the creative-gene prompt (see Figure 6A). Conditioned on PiP_i, GeneAgent autoregressively generates the serialized creative gene G~i=[gi,1,…,gi,Ti]\widetilde{G}_i = [g_{i,1}, \ldots, g_{i,T_i}]:

pθ(G~i∣Pi)=∏t=1Tipθ(gi,t∣Pi,gi,<t).(6)p_{\theta}(\widetilde{G}_i \mid P_i) = \prod_{t=1}^{T_i} p_{\theta}(g_{i,t} \mid P_i,g_{i,<t}). \tag*{(6)}

The supervised fine-tuning objective is

LSFT=−1∣DSFT∣∑(Pi,Gi)∈DSFT∑t=1Timi,t(7)\mathcal{L}_{\mathrm{SFT}} = -\frac{1}{|\mathcal{D}_{\mathrm{SFT}}|} \sum_{(P_i,G_i)\in\mathcal{D}_{\mathrm{SFT}}} \sum_{t=1}^{T_i} m_{i,t} \tag*{(7)}
log⁡pθ(gi,t∣Pi,gi,<t),\log p_{\theta}(g_{i,t} \mid P_i,g_{i,<t}),

where mi,tm_{i,t} masks the instruction tokens, textual product-attribute tokens, and visual tokens of the product images, so that the loss is computed only over the target gene tokens. We freeze the visual encoder and vision–language alignment module and optimize the LoRA parameters of the language model. The resulting SFT model initializes GRPO and serves as fixed reference policy for KL regularization.

CTR-Level Reward Modeling

Raw CTR distributions vary substantially across categories. We rank historical advertisements within each category by observed CTR and divide them into Low (bottom 30%), Mid (30–90%), and High (top 10%) levels. A frozen LightGBM [15] classifier is trained on handcrafted features ϕ(Pi,Gi)∈Rd\phi(P_i,G_i) \in\mathbb{R}^{d} (narrative-unit sequences, gene-rule keywords, TF–IDF of textual fields, and product attributes) to predict these CTR levels as a surrogate reward during GRPO training only (see Sec. A.3).

CTR-Guided GRPO

For each product input PiP_i, the old policy samples K=8K=8 candidate creative genes G^i(k)∼πθold(G∣Pi)\widehat{G}_i^{(k)} \sim\pi_{\theta_{\mathrm{old}}}(G \mid P_i). Each candidate is scored by a composite reward combining three terms: Ri(k)=0.60R~ctr,i(k)+0.25Rfmt,i(k)+0.15Rdiv,i(k)R_i^{(k)} = 0.60\widetilde{R}_{\mathrm{ctr},i}^{(k)} + 0.25R_{\mathrm{fmt},i}^{(k)} + 0.15R_{\mathrm{div},i}^{(k)}, where R~ctr\widetilde{R}_{\mathrm{ctr}} is the normalized CTR-level reward from the frozen surrogate, RfmtR_{\mathrm{fmt}} penalizes format violations (field completeness and serialization correctness), and RdivR_{\mathrm{div}} discourages repetitive candidates via Jaccard dissimilarity over narrative-unit sets and production-rule fields.

The GRPO objective maximizes the clipped surrogate with KL regularization against the frozen SFT reference policy, as shown in Eq. (8), where ρi,k,t\rho_{i,k,t} is the token-level importance ratio, A^i(k)\hat{A}_i^{(k)} is the group-relative advantage computed from the KK candidates of the same product, ϵ=0.2\epsilon=0.2 is the clipping coefficient, and β=0.04\beta=0.04 controls the KL penalty against the fixed SFT model. Full derivations of the reward components, advantage normalization, and KL term are provided in Sec. A.3.

LGRPO(θ)=−E[1K∑k=1K1Ti,k∑t=1Ti,k(min⁡[ρi,k,t(θ)A^i(k),clip⁡(ρi,k,t(θ),1−ϵ,1+ϵ)A^i(k)]−βDKLi,k,t)](8)\mathcal{L}_{\mathrm{GRPO}}(\theta)=-\mathbb{E}\left[\frac{1}{K}\sum_{k=1}^{K}\frac{1}{T_{i,k}}\sum_{t=1}^{T_{i,k}}\left(\min\left[\rho_{i,k,t}(\theta)\hat{A}_{i}^{(k)},\operatorname{clip}\left(\rho_{i,k,t}(\theta),1-\epsilon,1+\epsilon\right)\hat{A}_{i}^{(k)}\right]-\beta D_{\mathrm{KL}}^{i,k,t}\right)\right] \tag*{(8)}

VideoAgent

Given PiP_i and G~i\widetilde{G}_i, VideoAgent produces the advertising video as follows (Figure 2). Before expanding the gene into scripts, it retrieves category-then-gene constraints from the long-term memory E\mathcal{E} (detailed under Long-Term Memory below). It then executes shot-script planning and narration/subtitle adaptation, alongside clip retrieval, TTS synthesis, BGM matching, and timeline rendering.

Creative-Gene-Guided Shot Planning

Conditioned on the creative gene, and the retrieved constraints from Long-Term Memory and Reflection, VideoAgent expands the gene into an executable shot-by-shot script, yielding a storyboard Bi=[bi,1,…,bi,Li]B_i=[b_{i,1},\ldots,b_{i,L_i}], where each narrative unit maps to one representative shot with a visual description, voice-over text, duration, and production requirements. Narrative units determine shot goals and order; gene rules control script style, voice-over and music; product images keep shot descriptions appearance-consistent with the advertised product; and Eiret\mathcal{E}_i^{\mathrm{ret}} constrains error-prone choices during script expansion.

Video-Clip Retrieval, TTS, BGM, and Rendering

VideoAgent retrieves clips from MiM_i with Qwen3-VL-Embedding-8B [16]; CosyVoice 2 [8] synthesizes narration; BGM combines Qwen3-Embedding-8B matching with librosa tempo, energy, and spectral cues [26]; ffmpeg renders the timeline [35].

Reflection

Although the creative gene provides a global production plan, residual errors may still arise during execution. VideoAgent therefore performs a four-stage reflection loop on the rendered video: it evaluates the video along six dimensions (NQ, SPC, VSC, MQA, MSS, TB) with temporally grounded evidence, diagnoses deficient decisions by localizing defects to specific shots or production modules, revises only the affected components, and verifies the revised video under the same protocol until quality requirements are met or the maximum iteration count is reached (see Sec. B.5). Verified failure-correction experiences are written into the long-term memory E\mathcal{E} and later retrieved by category-then-gene matching to constrain script production for new genes.

Long-Term Memory

Long-term memory complements the reflection loop through a read–write mechanism for production experience. It is pre-populated with verified failure-correction experiences extracted from historical production cases in CutAgent-70K. Before a creative gene is expanded into shot scripts, VideoAgent reads relevant experiences from memory as production constraints. After rendering, the reflection loop diagnoses execution failures and writes newly verified corrections back into memory for future tasks. To reuse production experience across products without parameter updates, VideoAgent maintains E={E(c)}c∈C\mathcal{E}=\{\mathcal{E}^{(c)}\}_{c\in\mathcal{C}}, where C\mathcal{C} is the set of 43 product categories and E(c)\mathcal{E}^{(c)} stores experiences for category cc, disjoint from the clip pool MiM_i. Each entry e=(G~e,de,me,ce)e=(\widetilde{G}_e,d_e,m_e,c_e) stores a gene context G~e\widetilde{G}_e, an observed defect description ded_e, the affected production module mem_e, and a verified correction cec_e (e.g., “avoid repeated close-ups in the opening and insert a trust-building unit”).

Retrieval is performed in two stages before shot-script generation. First, given the product category cic_i, VideoAgent selects the corresponding shard E(ci)\mathcal{E}^{(c_i)} so that only experiences from the same category are candidates. Second, within E(ci)\mathcal{E}^{(c_i)}, it matches the current gene G~i\widetilde{G}_i to stored gene contexts with BGE-M3 [6]: both genes are serialized to text, embedded, and ranked by cosine similarity, yielding the top-KK experiences

Eiret=TopK⁡({e∈E(ci)},cos⁡(BGE-M3⁡(G~i),BGE-M3⁡(G~e))).(9)\mathcal{E}_i^{\mathrm{ret}}=\operatorname{TopK}\left(\left\{e\in\mathcal{E}^{(c_i)}\right\},\operatorname{cos}\left(\operatorname{BGE\text{-}M3}(\widetilde{G}_i),\operatorname{BGE\text{-}M3}(\widetilde{G}_e)\right)\right). \tag*{(9)}

The retrieved corrections are injected as explicit constraints that tell the planner what this gene should attend to when producing shot scripts—for example, avoiding redundant close-ups, covering omitted selling points, or preserving trust-building beats. Thus memory retrieval is category-then-gene conditioned and independent of running reflection on the current video; reflection on prior videos is what populates E\mathcal{E}.

Experiments

Implementation Details

GeneAgent uses Qwen3-VL-8B-Instruct. We freeze the visual encoder and vision–language alignment module, and apply LoRA to all linear layers with rank 16, scaling factor 32, and dropout 0.05. SFT trains on the top 20% CTR samples for three epochs with learning rate 5×10−55 \times10^{-5}, batch size 32, and sequence length 4,096. GRPO then uses all CutAgent-70K for one epoch with K=8K=8, learning rate 1×10−61 \times10^{-6}, clipping coefficient 0.2, and KL coefficient 0.04. The reward model uses 31 leaves and 500 boosting rounds. The loss is computed only over creative-gene tokens.

Baselines and Evaluation Protocol

To evaluate high-CTR e-commerce advertising video creation and the effectiveness of creative-gene-driven planning, we compare CutAgent with the end-to-end advertising video creation method AutoCut [46] and the agentic editing system FireRed-OpenStoryline [10]. FireRed-OpenStoryline is an open agentic editor that consumes user media and natural-language directives to search/organize clips, generate scripts, recommend music/voiceover, and render timelines; given the same product materials and an advertising-oriented prompt, it is a fair tool-orchestration baseline for end-to-end video production. To isolate gene generation quality, we further evaluate creative genes produced by Qwen3-VL-8B-Instruct [2], Seed-2.0-Pro [29], DeepSeek-V3 [19], Kimi-2.5 [34], Gemini-3-Pro [33], and ChatGPT-5.6-Sol [1], and convert all genes with the same VideoAgent. All methods receive the same multi-modal product inputs (attributes and images) and material pools on CutAgent-Eval and are assessed under an identical evaluation protocol.

Experimental Results

Evaluation Metrics

Narrative Quality (NQ) measures whether the video forms a complete, coherent, and engaging advertising narrative. It averages narrative completeness (logical organization, information density, opening/ending) and attractiveness (attention capture, rhythm, memorability), each scored in [0,50][0, 50].

Selling-Point Coverage (SPC) evaluates how completely the video communicates the core selling points of the target product. We first extract candidate selling points from the product detail page and retain only those that can be visually demonstrated or explicitly conveyed in a short advertising video. For each retained selling point, the evaluator examines the visual frames, narration, subtitles, and on-screen text and assigns a matching score mk∈[0,1]m_k \in[0, 1]. A selling point is regarded as covered when its matching score exceeds a fixed threshold τ\tau. Given KK retained selling points, SPC is calculated as SPC=1K∑k=1KI(mk≥τ)×100\mathrm{SPC} = \frac{1}{K}\sum_{k=1}^{K}\mathbb{I}(m_k \ge\tau) \times100.

Visual–Script Consistency (VSC) evaluates the semantic consistency between visuals and corresponding narration/subtitles. ASR-aligned transcript segments are compared with co-temporal frames by the multi-modal evaluator.

Material Quality Assessment (MQA) evaluates the clarity, relevance, and diversity of the visual materials, penalizing blur, noise, exposure issues, irrelevant content, and cross-shot redundancy.

Music Similarity Score (MSS) evaluates whether the background music is compatible with the product category, visual content, and editing rhythm. We infer the desired music style from frames and subtitles, separate the background music from speech, and measure compatibility via CLAP embedding cosine similarity:

sMSS=cos⁡(fCLAP(abgm),fCLAP(tstyle)).(10)s_{\mathrm{MSS}} = \cos\left(f_{\mathrm{CLAP}}(a_{\mathrm{bgm}}), f_{\mathrm{CLAP}}(t_{\mathrm{style}})\right). \tag*{(10)}

Trust Building (TB) evaluates whether the video makes the product appear reliable and worth purchasing, combining scene immersion (realistic usage scenarios) and trust endorsement (certifications, reviews, demonstrations), each scored in [0,50][0, 50].

Quantitative Comparisons

We evaluate all methods on CutAgent-Eval, as summarized in Table 1. All methods receive identical multi-modal product inputs and clip pools; gene-generation baselines share our VideoAgent for downstream production, while end-to-end baselines use their own editors. The generated videos are comprehensively assessed across three paradigms: a six-dimensional automated quality protocol (scored on a 0–100 scale using Qwen3.5-397B-A17B [2]; details in Sec. D), real-world online CTR, and a forced-choice human preference study. In particular, compared with AutoCut, which maps product inputs directly to video-production decisions without an explicit creative-gene planning stage, CutAgent attains consistently better automated quality, the highest scores in five of the six automated quality dimensions, the highest human preference (60.4%), and the best online CTR (2.21%).

MethodNQ↑SPC↑VSC↑MQA↑MSS↑TB↑Online CTR (%)↑A/B Pref. (%)↑
Creative-Gene-Driven Planning
Qwen3-VL-8B-Instruct86.879.683.284.785.466.81.1237.5
Seed-2.0-Pro94.182.784.185.885.970.61.3840.8
DeepSeek-V388.983.584.786.286.372.11.5139.7
Kimi-2.589.785.285.587.086.874.51.7842.3
Gemini-3-Pro90.186.185.987.587.176.21.0341.2
ChatGPT-5.6-Sol87.888.086.588.287.578.61.9144.1
Advertising Video Creation
AutoCut87.475.382.684.485.862.31.9834.6
FireRed-OpenStoryline90.271.280.682.190.268.21.8236.9
Ours (CutAgent)94.990.987.489.688.581.32.2160.4

Table 1. Quantitative comparison with creative-planning and advertising-video creation baselines on CutAgent-Eval.

Qualitative Comparison

Figure 5 in Sec. E compares the two systems shot by shot on two representative products. AutoCut opens with near-duplicate shots and flattens into a selling-point list, whereas CutAgent orders attention capture, product recognition, feature demonstration, and trust building, covering complementary product evidence rather than redundant close-ups, and raises the observed CTR from 1.83% to 2.89% and from 1.27% to 2.81%, suggesting that the gain stems from narrative ordering rather than from shot-level aesthetics.

Figure 5. Qualitative comparison between AutoCut and CutAgent on a glass teapot set and sandy-soil honey sweet potatoes. shot-level narratives of AutoCut (top) and CutAgent (bottom), the corresponding visual frames, and the narrative units realized by CutAgent, together with the observed CTR of each video.

Online CTR

To evaluate real-world advertising effectiveness, we ran a one-week online experiment on a large e-commerce platform covering 43 categories and 430 products (10 per category), with over 10 million ad impressions. All methods share same products and randomly allocated traffic; the LightGBM predictor of § 3.3.4 serves only as a training-time reward.

Human Evaluation

We conduct a forced-choice A/B study with 200 products sampled from CutAgent-Eval and 30 participants, collecting 1,600 pairwise judgments (200 products × 8 methods). Each baseline is compared with CutAgent on the same products, and CutAgent’s overall preference score averages its forced-choice win rate over the eight methods. Participants choose without ties, considering all six dimensions and overall appeal (details in Sec. D).

Ablation Study

Table 2 evaluates SFT, CTR-guided GRPO, reflection, and long-term memory; the Base variant uses Qwen3-VL-8B-Instruct without task-specific optimization. SFT substantially improves all metrics: NQ/SPC rise from 86.8/79.6 to 92.0/87.3, TB from 66.8 to 76.2, online CTR from 1.12% to 1.76%, and human preference from 37.5% to 54.2%. Reflection mainly improves narrative and post-production quality: SFT+Reflection reaches NQ 94.6, SPC 90.1, MQA 88.9, TB 81.5, preference 59.1%, and CTR 1.89%. GRPO instead improves advertising effectiveness and cross-modal alignment: SFT+GRPO reaches CTR 2.05%, with VSC/MSS increasing from 85.4/87.2 to 87.3/88.7. Combining both yields NQ 94.7, SPC 90.7, MQA 89.7, preference 59.8%, and CTR 2.14%. Long-term memory adds smaller but consistent gains: NQ, SPC, VSC, MSS, and TB improve by 0.2, 0.2, 0.2, 0.2, and 0.3 points, raising CTR and preference to 2.21% and 60.4%.

VariantSFTGRPOReflectionMemoryNQ↑SPC↑VSC↑MQA↑MSS↑TB↑Online CTR (%)↑A/B Pref. (%)↑
Base86.879.683.284.785.466.81.1237.5
SFT✓92.087.385.487.187.276.21.7654.2
SFT + Reflection✓✓94.690.186.788.988.181.51.8959.1
SFT + GRPO✓✓93.088.887.388.088.778.42.0557.9
SFT + GRPO + Reflection✓✓✓94.790.787.289.788.381.02.1459.8
SFT + GRPO + Reflection + Memory✓✓✓✓94.990.987.489.688.581.32.2160.4

Table 2. Ablation of SFT, GRPO, reflection, and long-term memory on CutAgent-Eval.

Conclusion

We presented CutAgent, a dual-agent framework that formulates e-commerce advertising video creation as a Plan–Execute–Self–Evolve process. GeneAgent maps multimodal product inputs—textual attributes together with product images—to structured creative genes through highCTR supervised fine-tuning and CTR-guided GRPO on CutAgent-70K, whose genes are recovered by reverse extraction from historical ads. VideoAgent converts these genes into complete advertising videos under category-then-gene constraints from a long-term memory E\mathcal{E}, and independently refines production decisions via reflection, writing verified corrections back to E\mathcal{E}. Experiments on CutAgent-Eval show that CutAgent consistently outperforms representative open-source and proprietary baselines in video quality, online CTR, and human preference, achieving the highest online CTR of 2.21%.

GeneAgent Training Details

Narrative Unit Registry

Table 3 presents the complete narrative-unit vocabulary used by CutAgent. The vocabulary contains 27 narrative units organized into six functional families: attention capture, product introduction, functional demonstration, trust building, promotional conversion, and closing.

Hyperparameters

We adopt Qwen3-VL-8B-Instruct as the backbone of GeneAgent. During both SFT and GRPO, each input is the multimodal product representation PiP_i: textual product attributes (category, SKU title, price, selling points, and review summary) together with the main and detail product images. The gene-generation instruction follows the creative-gene prompt in Figure 6(A).

We freeze the visual encoder and the vision–language alignment module, and apply LoRA to all linear layers of the language model. The LoRA rank, scaling factor, and dropout rate are set to 16, 32, and 0.05, respectively. GeneAgent is first supervised fine-tuned on the top 20% CTR samples of CutAgent-70K for three epochs using AdamW [22] with an initial learning rate of 5×10−55 \times10^{-5}, a weight decay of 0.01, and a cosine learning-rate schedule with a warm-up ratio of 0.05. The effective batch size is 32, the maximum sequence length is 4,096, and the maximum gradient norm is 1.0. We use BF16 precision [14], FlashAttention [7], and gradient checkpointing. The training loss is computed only over the target creative-gene tokens.

Starting from the SFT checkpoint, we further optimize GeneAgent with CTR-guided GRPO on the full CutAgent-70K for one epoch. For each product we sample K=8K=8 candidate genes with temperature 1.0 and top-p=0.9p=0.9, and score them with the composite reward. We use AdamW with learning rate 1×10−61 \times10^{-6}, effective batch size 32 (1 product ×\times 8 candidates per step when accumulated), clipping coefficient ϵ=0.2\epsilon=0.2, KL coefficient β=0.04\beta=0.04, and advantage stabilizer δ=10−6\delta=10^{-6}.

CTR-Level Reward Model Hyperparameters

We use the multiclass objective with learning rate 0.05, 31 leaves, maximum depth 8, 500 boosting rounds, and feature/sample subsampling ratios of 0.8. The minimum number of samples per leaf is 20, and inverse-frequency class weights alleviate imbalance.

We further evaluate the LightGBM-based CTR-level reward model used only during training in Table 4. The model achieves an accuracy of 87.6% on the CutAgent-70K training set and 76.9% on an independent held-out split from CutAgent-Eval product attributes and genes. Compared with the 60% majority-class baseline from the imbalanced CTR-level distribution, the reward model obtains a substantial improvement on unseen samples, indicating that it captures the relationship between product attributes, creative genes, and historical CTR levels for GRPO. This surrogate is never used to report the online CTR results in our main comparison or ablation tables.

DatasetSplitAccuracy (%)↑
CutAgent-70KTrain87.6
CutAgent-EvalTest76.9

Table 4. Accuracy of the training-time LightGBM CTR-level reward model on CutAgent-70K and a held-out gene-level test split.

CTR-Level Discretization

Raw CTR distributions vary substantially across categories. We rank historical advertisements separately within each category and divide them into Low-, Mid-, and High-CTR levels:

zi={low,yi<Q0.30(ci),mid,Q0.30(ci)≤yi<Q0.90(ci),high,yi≥Q0.90(ci).(11)z_i = \begin{cases} \mathrm{low}, & y_i < Q^{(c_i)}_{0.30},\\ \mathrm{mid}, & Q^{(c_i)}_{0.30} \le y_i < Q^{(c_i)}_{0.90},\\ \mathrm{high}, & y_i \ge Q^{(c_i)}_{0.90}. \end{cases} \tag*{(11)}

where Qα(ci)Q^{(c_i)}_{\alpha} is the α\alpha-quantile of CTR within category cic_i. This produces approximate proportions of 30%, 60%, and 10%.

Feature Construction and Reward Model

For each historical product–gene pair, we construct

xi=ϕ(Pi,Gi)∈Rd,d≈250.(12)\mathbf{x}_i = \phi(P_i,G_i) \in\mathbb{R}^{d}, \qquad d \approx250. \tag*{(12)}

The features include the narrative-unit sequence, keywords from all gene rules, TF–IDF representations of textual fields, and product attributes such as category and price. A LightGBM [15] classifier estimates

qψ(z∣Pi,Gi)=qψ(z∣ϕ(Pi,Gi)),(13)q_{\psi}(z \mid P_i,G_i) = q_{\psi}\left(z \mid\phi(P_i,G_i)\right), \tag*{(13)}

using the multiclass objective

LRM=−1N∑i=1Nlog⁡qψ(zi∣ϕ(Pi,Gi)).(14)\mathcal{L}_{\mathrm{RM}} = -\frac{1}{N}\sum_{i=1}^{N}\log q_{\psi}\left(z_i \mid\phi(P_i,G_i)\right). \tag*{(14)}

The trained classifier is frozen throughout GRPO.

Reward Components

For each product input PiP_i (textual attributes and product images), the old policy samples KK candidate creative genes:

G^i(k)∼πθold(G∣Pi),k=1,…,K.(15)\hat{G}^{(k)}_i \sim\pi_{\theta_{\mathrm{old}}}(G \mid P_i), \qquad k=1,\ldots,K. \tag*{(15)}

During training, the frozen CTR-level predictor assigns each candidate a CTR-level class:

z^i(k)=arg⁡max⁡z∈Z qψ(z∣ϕ(Pi,G^i(k))),(16)\hat{z}^{(k)}_i = \underset{z \in\mathcal{Z}}{\arg\max}\ q_{\psi}\left(z \mid\phi\left(P_i,\hat{G}^{(k)}_i\right)\right), \tag*{(16)}

where Z={low,mid,high}\mathcal{Z}=\{\mathrm{low},\mathrm{mid},\mathrm{high}\}.

The three levels receive rewards of 0, 2, and 5:

Rctr,i(k)={0,z^i(k)=low,2,z^i(k)=mid,5,z^i(k)=high.(17)R^{(k)}_{\mathrm{ctr},i} = \begin{cases} 0, & \hat{z}^{(k)}_i=\mathrm{low},\\ 2, & \hat{z}^{(k)}_i=\mathrm{mid},\\ 5, & \hat{z}^{(k)}_i=\mathrm{high}. \end{cases} \tag*{(17)}

We normalize this reward as

R~ctr,i(k)=Rctr,i(k)5.(18)\widetilde{R}^{(k)}_{\mathrm{ctr},i} = \frac{R^{(k)}_{\mathrm{ctr},i}}{5}. \tag*{(18)}

A format-compliance reward Rfmt,i(k)∈[0,1]R^{(k)}_{\mathrm{fmt},i} \in[0,1] evaluates field completeness, serialization correctness, and narrative-unit consistency. A gene-diversity reward discourages repetitive candidates:

Rdiv,i(k)=1−1K−1∑j=1j≠kKsim⁡(G^i(k),G^i(j)),(19)R^{(k)}_{\mathrm{div},i} = 1 - \frac{1}{K-1}\sum_{\substack{j=1 \\ j\ne k}}^{K}\operatorname{sim}\left(\widehat{G}^{(k)}_i,\widehat{G}^{(j)}_i\right), \tag*{(19)}

where sim⁡(⋅,⋅)\operatorname{sim}(\cdot,\cdot) is the mean Jaccard similarity over narrative-unit sets and discrete production-rule fields.

Group-Relative Advantage and Policy Optimization

For candidates generated from the same product, the group-relative advantage is

A^i(k)=Ri(k)−μiσi+δ,(20)\widehat{A}^{(k)}_i = \frac{R^{(k)}_i-\mu_i}{\sigma_i+\delta}, \tag*{(20)}

where

μi=1K∑k=1KRi(k),σi=1K∑k=1K(Ri(k)−μi)2.(21)\mu_i = \frac{1}{K}\sum_{k=1}^{K}R^{(k)}_i,\qquad\sigma_i = \sqrt{\frac{1}{K}\sum_{k=1}^{K}\left(R^{(k)}_i-\mu_i\right)^2}. \tag*{(21)}

Let g^i,k,t\widehat{g}_{i,k,t} denote the tt-th token of G^i(k)\widehat{G}^{(k)}_i. The token-level policy ratio is

ρi,k,t(θ)=πθ(g^i,k,t∣Pi,G^i,<t(k))πθold(g^i,k,t∣Pi,G^i,<t(k)).(22)\rho_{i,k,t}(\theta) = \frac{\pi_{\theta}\left(\widehat{g}_{i,k,t}\mid P_i,\widehat{G}^{(k)}_{i,<t}\right)}{\pi_{\theta_{\mathrm{old}}}\left(\widehat{g}_{i,k,t}\mid P_i,\widehat{G}^{(k)}_{i,<t}\right)}. \tag*{(22)}

The KL term is computed against the frozen SFT reference policy:

DKLi,k,t=DKL(πθ(⋅∣Pi,G^i,<t(k))∥πref(⋅∣Pi,G^i,<t(k))),(23)D^{i,k,t}_{\mathrm{KL}} = D_{\mathrm{KL}}\left(\pi_{\theta}\left(\cdot\mid P_i,\widehat{G}^{(k)}_{i,<t}\right)\middle\|\pi_{\mathrm{ref}}\left(\cdot\mid P_i,\widehat{G}^{(k)}_{i,<t}\right)\right), \tag*{(23)}

where πθold\pi_{\theta_{\mathrm{old}}} generates the current candidate group, while πref\pi_{\mathrm{ref}} denotes the fixed SFT model. δ=10−6\delta=10^{-6} stabilizes the group-relative advantage.

VideoAgent Production and Reflection Details

Guided by the creative gene, VideoAgent retrieves experiences from the long-term memory E\mathcal{E} by first selecting the product’s category shard among the 43 categories, then ranking gene contexts inside that shard with BGE-M3 [6] and injecting the top-KK matches as constraints for script expansion. The VideoAgent LLM prompts in Figure 6(B) are executed by Qwen3-235B-A22B [40]: B1 expands the gene into a shot-by-shot script, and B2 adapts narration and subtitles for TTS. Video-clip retrieval uses Qwen3-VL-Embedding-8B [16] over representative frames of candidate clips. TTS is performed with CosyVoice 2 [8], and word-level timestamps construct subtitle events. BGM retrieval combines Qwen3-Embedding-8B [16] semantic matching with acoustic features extracted by librosa [26], including tempo, energy, and spectral statistics. The final timeline is rendered with ffmpeg [35]. After rendering, VideoAgent runs the independent reflection loop (evaluation–diagnosis–revision–verification) for at most three rounds and writes verified failure–correction pairs back to E\mathcal{E} for subsequent category-then-gene retrieval.

Video-Clip Retrieval

For each shot, candidate clips are retrieved from the product-specific cleaned material pool Mi\mathcal{M}_i by encoding representative frames with a vision–language embedding model [16] and ranking by similarity to the shot description, while encouraging cross-shot visual diversity. Because the shot description is planned from PiP_i, including product images, retrieval favors clips that match both the narrative intent and the product appearance.

TTS and Subtitle Generation

Following the B2 prompt in Figure 6, VideoAgent adapts the shot scripts and voice-over rule into TTS-ready narration and subtitle styles, then synthesizes speech; word-level timestamps construct subtitle events. The script is shortened when its duration is inconsistent with the selected clip. Subtitle images are burned in at render time; unifying subtitle synthesis with visual text generation and editing, as in In-noText [20], is a direct route to more expressive on-screen typography Sec. H.

BGM and Sound-Effect Matching

Background music is retrieved by the music rule using semantic similarity and acoustic cues (tempo, energy, spectrum, emotion). Sound effects follow shot actions and the sound-effect rule.

Timeline Rendering

Selected clips, narration, subtitles, BGM, sound effects, and transitions are aligned on a timeline and rendered into the final video. Retrieved clips could in principle be replaced or supplemented by synthesized shots; doing so without subject drift or abrupt transitions is exactly the setting studied by FancyVideo [9] and RelaCtrl [5] (Sec. H).

Reflection Loop

VideoAgent performs a four-stage reflection loop on the rendered video.

Evaluation. The rendered video is evaluated from six fine-grained dimensions: narrative quality (NQ), selling-point coverage (SPC), visual–script consistency (VSC), material quality assessment (MQA), music similarity score (MSS), and trust building (TB). In addition to scalar scores, the evaluator produces temporally grounded evidence for individual shots, texts, audio segments, and editing elements.

Diagnosis. For each dimension that fails to satisfy its quality requirement, VideoAgent analyzes the evaluation evidence and localizes the defect to a specific temporal interval, shot instruction, or production module, such as a repeated clip, omitted selling point, mismatched narration, low-quality material, unsuitable music, or abrupt transition.

Revision. Based on the diagnosis, VideoAgent generates targeted correction instructions and updates the corresponding shot script. Rather than regenerating the entire video, it preferentially modifies only the affected components—for example, replacing a clip, rewriting a voice-over segment, adjusting shot duration, or rematching the background music—and re-executes the relevant production modules.

Verification. The revised video is evaluated again under the same protocol. If one or more quality requirements remain unsatisfied, the updated evidence is passed to the next diagnosis–revision cycle. Otherwise, the corrected video is accepted, and any verified failure–correction pair is written into the long-term memory for reuse on subsequent genes.

Dataset Construction Details

Category Distribution. CutAgent-70K covers 43 product categories organized into 8 portfolios, as shown in Figure 4: 3C & Digital, Appliance & Home, Household, Beauty & Fashion, Food & Beverage, Baby, Pets & Toys, Health, and Lifestyle. The outer radial bars indicate relative category mass within each portfolio, while the inner ring reports portfolio-level mean share, showing broad but non-uniform coverage across major e-commerce verticals.

Material Cleaning Pipeline. To support VideoAgent retrieval, historical advertising videos are processed by a lightweight material-cleaning pipeline into a reusable clip pool:

  1. Scene detection: ffmpeg splits each video into shots based on scene changes.

  2. Quality filtering: clips containing on-camera humans or lasting under 2 s are removed.

  1. Subtitle/watermark removal: overlaid subtitles and watermarks are detected and blurred while preserving on-product printed text.

  2. Deduplication: near-duplicate clips are eliminated by perceptual hashing.

After cleaning, each product typically yields 90–120 usable clips, denoted by Mi\mathcal{M}_i.

Reverse Gene Extraction. Historical creative genes are recovered from the original advertising videos by the reverse-extraction pipeline in Figure 6(C), which produces the supervision targets GiG_i of CutAgent-70K (distinct from the runtime GeneAgent prompt in Figure 6(A)).

Voice attribute analysis (C1) and BGM style analysis (C2) use Qwen3.5-Omni-Plus-Realtime on separated audio to obtain vocal attributes (e.g., gender, pitch, speed, expression) and a compact BGM style summary; these audio attributes later ground the voice-over rule rv,ir_{v,i} and music rule rm,ir_{m,i}.

Shot-level multimodal analysis (C3) uses Doubao-Seed-2.0-pro [29] (vision–language) on each shot and outputs visual content, oral content, and narrative purpose, with voice-over as the primary cue and visuals as auxiliary evidence.

Creative gene reconstruction (C4) uses Doubao-Seed-2.0-pro (text) with the shot-level analysis, the narrative-unit registry, and the audio attributes to restore a full creative gene in the gene schema, including the ordered narrative units Si=[si,1,…,si,ni]S_i = [s_{i,1}, \ldots, s_{i,n_i}] (si,j∈Ns_{i,j} \in\mathcal{N}) and the production rules rv,i,rt,i,rm,i,rs,i,rf,ir_{v,i}, r_{t,i}, r_{m,i}, r_{s,i}, r_{f,i}. Assembling these fields gives Gi=(Si,rv,i,rt,i,rm,i,rs,i,rf,i)G_i = (S_i, r_{v,i}, r_{t,i}, r_{m,i}, r_{s,i}, r_{f,i}).

Evaluation Metric Definitions

Narrative Quality (NQ)

NQ measures whether the final video forms a complete, coherent, and engaging advertising narrative rather than a simple concatenation of product materials. Narrative completeness evaluates the clarity of the narrative progression, logical organization of product information, information density, and the naturalness of the opening and ending. Attractiveness evaluates whether the opening captures attention, whether the intermediate content maintains an appropriate rhythm, and whether the video contains memorable or appealing elements. Each aspect is scored from 0 to 50, and NQ is computed as

NQ=Scompleteness+Sattractiveness,(24)\mathrm{NQ} = S_{\mathrm{completeness}} + S_{\mathrm{attractiveness}}, \tag*{(24)}

where Scompleteness,Sattractiveness∈[0,50]S_{\mathrm{completeness}, S_{\mathrm{attractiveness}} \in[0, 50]}.

Visual–Script Consistency (VSC)

Inspired by the Visual–Script Correlation metric in AutoCut [46], we adapt it to reference-free evaluation of complete advertising videos. We apply automatic speech recognition to obtain timestamped narration transcripts, align each text segment with the visual frames in the corresponding temporal interval, and ask the multimodal evaluator to assess whether the visual content conveys the expressed meaning and whether subtitles reinforce the visual focus. These aspects are jointly considered to produce a VSC score between 0 and 100. Because spatial reasoning remains a weak point of current MLLMs [23], VSC is best read as a relative signal between methods rather than an absolute measurement, and we therefore report it together with the forced-choice human study of Sec. D.7.

Material Quality Assessment (MQA)

MQA evaluates the quality, relevance, and diversity of the visual materials used in the final video, including image clarity, exposure quality, positive product presentation, relevance to the target product and its selling points, and unnecessary repetition across clips. Representative frames from each detected shot are examined for blur, noise, over-/underexposure, irrelevant content, and redundant materials. The evaluator then assigns an overall score between 0 and 100.

Music Similarity Score (MSS)

Following AutoCut [46], we adapt it to reference-free music–video evaluation. We use a multimodal evaluator to infer the desired music style from frames and subtitles, separate the background music from speech, and encode both the extracted music and the style description with CLAP [39]. The compatibility score is mapped to the 0–100 range with the same rule for all methods.

Trust Building (TB)

TB evaluates whether the advertising video makes the product appear reliable and worth purchasing. Scene immersion measures whether the video presents realistic usage scenarios and processes that viewers can imagine themselves in. Trust endorsement measures supporting evidence such as certifications, reviews, sales statistics, or real-person demonstrations. Each aspect is scored from 0 to 50, and

TB=Sscene+Sendorsement,(25)\mathrm{TB} = S_{\mathrm{scene}} + S_{\mathrm{endorsement}}, \tag*{(25)}

where Sscene,Sendorsement∈[0,50]S_{\mathrm{scene}, S_{\mathrm{endorsement}} \in[0, 50]}.

Online CTR

For method mm, online CTR is computed from the observed clicks and impressions:

CTRm=∑i=1NCim∑i=1NEim×100%,(26)\mathrm{CTR}_m = \frac{\sum_{i=1}^{N} C_i^m}{\sum_{i=1}^{N} E_i^m} \times100\%, \tag*{(26)}

where CimC_i^m and EimE_i^m denote the numbers of clicks and impressions received by the video generated for product ii, respectively.

Human Evaluation

For a competing method mm,

Pref.(m)=NmNm+NCutAgent×100%,(27)\mathrm{Pref.}(m) = \frac{N_m}{N_m + N_{\mathrm{CutAgent}}} \times100\%, \tag*{(27)}

where NmN_m and NCutAgentN_{\mathrm{CutAgent}} denote the numbers of votes received by method mm and CutAgent, respectively. Since each comparison uses a forced-choice protocol, the two preference rates sum to 100%. When all competing methods are evaluated using the same number of pairwise judgments, the overall preference score of CutAgent is

Pref.(CutAgent)=1M∑m=1M(100%−Pref.(m)),(28)\mathrm{Pref.}(\mathrm{CutAgent}) = \frac{1}{M} \sum_{m=1}^{M} (100\% - \mathrm{Pref.}(m)), \tag*{(28)}

where MM denotes the number of competing methods.

Qualitative Analysis

We provide a detailed shot-by-shot comparison between AutoCut and CutAgent on two representative products from CutAgent-Eval. We choose AutoCut as the qualitative baseline because it is the most directly comparable end-to-end method for e-commerce advertising video creation: both systems consume product-related materials and output complete videos. For each product, Figure 5 shows the product attributes (title, category, price, and customer feedback), the product at:VEVENT shot-level narratives of AutoCut (top) and CutAgent (bottom), the corresponding visual frames, and the narrative units realized by CutAgent, together with the observed CTR of each video.

Glass Teapot Set. In the glass teapot example (1000 mL Glass Teapot Set; category: Tea Set; price: 12.99 RMB), customer feedback emphasizes style, durability, heat resistance, easy cleaning, and cost-effectiveness. AutoCut opens with two highly similar shots that repeatedly present the product under nearly identical viewpoints (0–3 s), which wastes early attention and reduces visual diversity. Subsequent shots mention a stainless-steel filter and detachable accessories, but the overall storyline remains close to a flat selling-point list. By contrast, CutAgent organizes the advertisement as a conversion-oriented narrative sequence. It first captures everyday friction with attention capture (NU−D3: messy or inconvenient ceramic tea sets), then reveals the solution through product recognition (NU−C1), demonstrates detachable design and usage scenarios with feature-demonstration units (NU−H9, NU−Z2), and finally consolidates purchase confidence via trust building (NU−V4: quality assurance and reviews). The resulting video covers complementary aspects of the product rather than redundant close-ups, and the observed CTR increases from 1.83% (AutoCut) to 2.89% (CutAgent).

Sandy-Soil Honey Sweet Potatoes. In the sandy-soil honey sweet potatoes example (category: Sweet Potato; price: 2.99 RMB / 2 jin), customer feedback highlights sweetness and delivery speed, while also mentioning variability in freshness and weight. AutoCut begins with an appealing melted-center shot, but Shot 4 (“advantages at a glance”) uses a comparatively low-quality and less informative frame, weakening mid-video engagement before the final call to action. CutAgent instead follows a gene-conditioned progression: strong visual impact for attention capture (NU–P0), origin traceability for trust building (NU–K3: sandy-soil farming and on-site harvesting), close-up and overview for product recognition (NU–C1, NU–Q7), usage-scenario demonstration (NU–Z2), and quality assurance (NU–V4). The retrieved materials remain clearer and more semantically aligned with each narrative unit. Correspondingly, the observed CTR rises from 1.27% (AutoCut) to 2.81% (CutAgent).

Prompt Templates

Figure 6 presents the complete prompt templates used across the CutAgent pipeline. We describe each group below.

(A) Creative-Gene Generation. This prompt is used by GeneAgent at both training time (to define the SFT target format) and inference time (to generate creative genes for new products). Given the multimodal product input PiP_i—textual attributes (category, SKU title, price, selling points, review summary) together with the main and detail product images—the prompt instructs GeneAgent to output a structured creative gene comprising an ordered narrative-unit sequence Si\mathcal{S}_i drawn from the vocabulary NU\mathcal{N}\mathcal{U}, along with production rules for script style (rs,ir_{s,i}), voice-over (rv,ir_{v,i}), transitions (rt,ir_{t,i}), background music (rm,ir_{m,i}), and sound effects (r~f,i)(\widetilde{r}_{f,i}).

(B) VideoAgent Prompts. B1B1 (Shot-Script Planning) takes the generated creative gene G~i\widetilde{G}_i, the multimodal product input PiP_i, and the retrieved failure–correction constraints Eiret\mathcal{E}^{\mathrm{ret}}_i as context, and instructs VideoAgent to expand the gene into an executable shot-by-shot storyboard BiB_i, specifying visual descriptions, voice-over text, duration, and production requirements for each shot. B2B2 (Narration and Subtitle Adaptation) takes the expanded shot scripts and the voice-over rule r~v,i\widetilde{r}_{v,i}, and rewrites the scripts into TTS-ready narration with appropriate subtitle styles, adjusting length when clip duration constraints apply.

(C) Reverse-Extraction Prompts for CutAgent-70K. These prompts recover creative genes from historical advertising videos to construct supervision targets. C1C1 (Voice Attribute Analysis) extracts vocal characteristics (gender, pitch, speed, expression) from the separated speech track. C2C2 (BGM Style Analysis) summarizes the background music style (tempo, energy, emotion, genre) from the separated music track. C3C3 (Shot-Level Multimodal Analysis) processes each detected shot to identify its visual content, oral content (voice-over transcript), and narrative purpose, using voice-over as the primary cue. C4C4 (Creative Gene Reconstruction) synthesizes the shot-level analyses, audio attributes from C1–C2, and the narrative-unit registry into a complete creative gene in the target schema.

System Demonstration

Beyond offline evaluation, we package CutAgent as an interactive workbench for operators who create e-commerce advertising videos at scale. As shown in Figure 7, the interface follows the same Plan–Execute–Self-Evolve pipeline as our method. On the left, users load a multimodal product representation PiP_i (main/detail images and textual attributes) and invoke GeneAgent to generate a structured creative gene (Figure 6(A)). The center hub visualizes narrative units and production rules in G~i\widetilde{G}_i, allowing light manual inspection before rendering. On the right, VideoAgent retrieves category-then-gene constraints from E\mathcal{E}, then executes gene-guided shot planning, clip retrieval, TTS, BGM matching, and timeline rendering (Figure 6(B)). The bottom Reflection panel reports the six evaluation dimensions (NQ, SPC, VSC, MQA, MSS, TB) with temporally grounded diagnoses, so operators can accept automatic revisions or intervene when needed; verified corrections are written back to E\mathcal{E}. The glass teapot example in the figure is the same product analyzed in Figure 5.

CutAgent system interface for e-commerce advertising video creation

Figure 7. CutAgent system interface for e-commerce advertising video creation. From left to right, the workbench exposes multimodal product attributes PiP_i (main/detail images and textual metadata), the GeneAgent hub that edits the structured creative gene G~i\widetilde{G}_i (narrative units and production rules), and the VideoAgent panel for memory-constrained shot planning, clip retrieval, TTS, BGM matching, and rendering. The bottom Reflection panel visualizes the six-dimensional evaluation (NQ, SPC, VSC, MQA, MSS, TB) together with temporally grounded diagnosis and revision, closing the Plan–Execute–Self-Evolve loop.

Related Work: Generative Video Models

§ 2 reviews retrieval- and template-based pipelines together with agentic editing systems. A second and rapidly moving line of work generates the frames themselves instead of assembling existing footage, and it is directly relevant to the production modules of CutAgent: a shot whose retrieved material is missing or unusable cannot be produced at all, and on-screen text has to be composed rather than overlaid. We group representative methods of this line by the property they contribute.

Layout and Narrative-Controllable Generation

Lay2Story [24] conditions a diffusion transformer on an explicit layout so that a multi-panel story keeps its subjects consistent, and PlanGen [12] goes further by planning the spatial layout inside an auto-regressive vision–language model before any pixel is generated. RelaCtrl [5] shows that control signals contribute very unevenly across layers and that routing them by relevance keeps control cheap. These results are consistent with the premise behind GeneAgent: structure should be decided explicitly before content is produced, and it should be compact enough to condition generation on. The two families differ mainly in the medium of the structure. For these generators the structure is a latent layout condition consumed by the generator, whereas for CutAgent the same prior decision is written as an editable production plan—shot instructions, retrieval queries, narration, subtitle and music rules—that can be scored against a business objective and revised in place.

Subject, Style, and Physical Consistency

MoFu [18] preserves the scale and identity of several reference subjects through scale-aware modulation and Fourier fusion, and removes the sensitivity to reference ordering. U-StyDiT [43] repaints ultra-high-resolution images in an arbitrary artistic style with a diffusion transformer, while WISA [37] injects explicit physical guidance so that a text-to-video model respects physical laws instead of producing plausible-looking motion. CustomTTT [4] customizes appearance and motion jointly by test-time training, and FancyVideo [9] replaces frame-agnostic text control with cross-frame textual guidance, which improves both dynamics and temporal consistency over long durations. These methods target the same failure mode: divergence of subject identity, style, or physical behaviour as a synthesized video grows longer.

Retrieval avoids most of it by construction, since the advertised product is filmed rather than re-imagined, but it pays in coverage—if the material for a required shot is missing or unusable, the shot cannot be produced at all. These generators are therefore the natural synthesizers behind VideoAgent’s shot plan (Sec. B): the plan already names the subject, the style, and the behaviour a shot needs, which is precisely the conditioning these models consume. Drift also accumulates when no control signal is involved at all: the multi-step samplers of a flow-matching backbone amplify the discrepancy between the training and the inference trajectory. DEFAR [13] turns this exposure bias into its own correction signal, deriving directional and frequency-adaptive feedback from a simulated single-step inference, and thereby raises the model’s tolerance to its own deviation. Rectification at this backbone level complements the control-side methods above.

Visual Text and Spatial Reasoning

Advertising video carries text—selling points, prices, call-to-action lines—whose legibility matters as much as the footage. InnoText [20] unifies visual text generation and editing in a single model and targets the structural regularity that small and non-Latin glyphs demand, which is where the render-time subtitle burn-in of Sec. B is weakest; a shot-local text renderer is a direct upgrade that leaves the rest of the pipeline untouched. On the evaluation side, Lu et al. [23] find that leading MLLMs fail textbook spatial-reasoning problems by a wide margin, so a score produced by an MLLM evaluator is best read as a relative signal between methods rather than an absolute measurement of quality, which is why the metric suite of Sec. D is always reported together with the forced-choice human study of Sec. D.7.

References

  1. [1]Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmadi, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023.arxiv.org/abs/2303.08774
  2. [2]Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjang Ge, et al. Qwen3-vl technical report. arXiv preprint arXiv:2511.21631, 2025.arxiv.org/abs/2511.21631
  3. [3]Max Bain, Arsha Nagrani, Gül Varol, and Andrew Zisserman. Frozen in time: A joint video and image encoder for end-to-end retrieval. In Proceedings of the IEEE/CVF international conference on computer vision, pages 1728–1738, 2021.arxiv.org/abs/2104.00650
  4. [4]Xiuli Bi, Jian Lu, Bo Liu, Xiaodong Cun, Yong Zhang, Weisheng Li, and Bin Xiao. CustomTTT: Motion and appearance customized video generation via test-time training. In Proceedings of the AAAI Conference on Artificial Intelligence, 2025.arxiv.org/abs/2412.15646
  5. [5]Ke Cao, Jing Wang, Ao Ma, Jiasong Feng, Xuanhua He, Run Ling, Haowei Liu, Jian Lu, Wei Feng, Haozhe Wang, Hongjuan Pei, Yihua Shao, Zhanjie Zhang, and Jie Zhang. RelaCtrl: Relevance-guided efficient control for diffusion transformers. In Proceedings of the AAAI Conference on Artificial Intelligence, 2026.
  6. [6]Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. Bge m3-embedding: Multilingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. arXiv preprint arXiv:2402.03216, 4(5), 2024.arxiv.org/abs/2402.03216
  7. [7]Tri Dao, Daniel Y Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. Flashattention: Fast and memory-efficient exact attention with io-awareness. In Advances in neural information processing systems, 2022.arxiv.org/abs/2205.14135
  8. [8]Zhihao Du, Yuxuan Wang, Qian Chen, Xian Shi, Xiang Lv, Tianyu Zhao, Zhifu Gao, Yexin Yang, Changfeng Gao, Hui Wang, et al. Cosyvoice 2: Scalable streaming speech synthesis with large language models. arXiv preprint arXiv:2412.10117, 2024.arxiv.org/abs/2412.10117
  9. [9]Jiasong Feng, Ao Ma, Jing Wang, Ke Cao, and Zhanjie Zhang. FancyVideo: Towards dynamic and consistent video generation via cross-frame textual guidance. In Proceedings of the International Joint Conference on Artificial Intelligence, 2025.arxiv.org/abs/2408.08189
  10. [10]FireRed Team. FireRed-OpenStoryline: An agentic framework for autonomous, human-aligned video creation. https://github.com/FireRedTeam/FireRed-OpenStoryline, 2026. Accessed: 2026-07-18.
  11. [11]Huan-gang Gao, Jiayi Geng, Wenyue Hua, Mengkang Hu, Xinzhe Juan, Hongzhang Liu, Shilong Liu, Jiahao Qiu, Xuan Qi, Yiran Wu, et al. A survey of self-evolving agents: On path to artificial super intelligence. arXiv e-prints, pages arXiv–2507, 2025.
  12. [12]Runze He, Bo Cheng, Yuhang Ma, Qingxiang Jia, Shanyuan Liu, Ao Ma, Xiaoyu Wu, Liebucha Wu, Dawei Leng, and Yuhui Yin. PlanGen: Towards unified layout planning and image generation in auto-regressive vision language models. arXiv preprint arXiv:2503.10127, 2025.arxiv.org/abs/2503.10127
  13. [13]Guanbo Huang, Jingjia Mao, Fanding Huang, Fengkai Liu, Xiangyang Luo, Yaoyuan Liang, Jiasheng Lu, Xiaoe Wang, Pei Liu, Ruiliu Fu, Ruqi Huang, and Shao-Lun Huang. Exposure bias can alleviate itself via directional and frequency rectification in flow matching. arXiv preprint arXiv:2606.28226, 2026.arxiv.org/abs/2606.28226
  14. [14]Dhiraj Kalamkar, Dheevatsa Mudigere, Naveen Mellem­pudi, Dipankar Das, Kunal Banerjee, Sasikanth Avan­cha, Dharma Teja Vooturi, Nataraj Jammalamadaka, Jianyu Huang, Hector Yuen, et al. A study of bfloat16 for deep learning training. arXiv preprint arXiv:1905.12322, 2019.arxiv.org/abs/1905.12322
  15. [15]Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. Lightgbm: A highly efficient gradient boosting decision tree. Advances in neural information processing systems, 30, 2017.
  16. [16]Mingxin Li, Yanzhao Zhang, Dingkun Long, Keqin Chen, Sibo Song, Shuai Bai, Zhibo Yang, Pengjun Xie, An Yang, Daiheng Liu, et al. Qwen3-vl-reranker: A unified framework for state-of-the-art multimodal retrieval and ranking. arXiv preprint arXiv:2601.04720, 2026.arxiv.org/abs/2601.04720
  17. [17]Han Lin, Abhay Zala, Jaemin Cho, and Mohit Bansal. Videoredirectgpt: Consistent multi-scene video generation via llm-guided planning. arXiv preprint arXiv:2309.15091, 2023.arxiv.org/abs/2309.15091
  18. [18]Run Ling, Ke Cao, Jian Lu, Ao Ma, Haowei Liu, Runze He, Changwei Wang, Rongtao Xu, Yihua Shao, Zhanjie Zhang, Peng Wu, Guibing Guo, Wei Feng, Zheng Zhang, Jingjing Lv, Junjie Shen, Ching Law, and Xingwei Wang. MoFu: Scale-aware modulation and fourier fusion for multi-subject video generation. In Proceedings of the AAAI Conference on Artificial Intelligence, 2026.
  19. [19]Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024.arxiv.org/abs/2412.19437
  20. [20]Haowei Liu, Runze He, Jian Lu, Ao Ma, Run Ling, Ke Cao, Jiasong Feng, Wei Feng, Shuo Lu, Yexing Xu, Yun Wang, Jing Wang, and Zhanjie Zhang. InnoText: A unified model for visual text generation and editing. In Proceedings of the European Conference on Computer Vision, 2026.arxiv.org/abs/2607.22101
  21. [21]Xianjie Liu, Yiman Hu, Liang Wu, Ping Hu, Yixiong Zou, Jian Xu, and Bo Zheng. E-vads: An e-commerce short videos understanding benchmark for mlms. arXiv preprint arXiv:2602.08355, 2026.arxiv.org/abs/2602.08355
  22. [22]Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017.arxiv.org/abs/1711.05101
  23. [23]Shuo Lu, Jianjie Cheng, Yinuo Xu, Yongcan Yu, Lijun Sheng, Peijie Wang, Siru Jiang, Yongquan Hu, Run Ling, Yihua Shao, Ao Ma, Wei Feng, Lingxiao He, Meng Wang, Qianlong Xie, Xingxing Wang, Nicu Sebe, Ran He, and Jian Liang. Do MLLMs really understand space? A mathematical reasoning evaluation. arXiv preprint arXiv:2602.11635, 2026.
  24. [24]Ao Ma, Jiasong Feng, Ke Cao, Jing Wang, Yun Wang, Qunwei Zhang, and Zhanjie Zhang. Lay2Story: Extending diffusion transformers for layout-togglable story generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2025.arxiv.org/abs/2508.08949
  25. [25]Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Khan. Video-chatgpt: Towards detailed video understanding via large vision and language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12585–12602, 2024.arxiv.org/abs/2306.05424
  26. [26]Brian McFee, Colin Raffel, Dawen Liang, Daniel PW Ellis, Matt McVicar, Eric Battenberg, Oriol Nieto, et al. librosa: Audio and music signal analysis in python. SciPy, 2015(18-24):7, 2015.DOI
  27. [27]Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730–27744, 2022.arxiv.org/abs/2203.02155
  28. [28]Dongjun Qian, Kai Su, Yiming Tan, Qishuai Diao, Xian Wu, Chang Liu, Bingyue Peng, and Zehuan Yuan. Vc-llm: Automated advertisement video creation from raw footage using multi-modal llms. arXiv preprint arXiv:2504.05673, 2025.arxiv.org/abs/2504.05673
  29. [29]Bytedance Seed. Seed2. 0 model card: Towards intelligence frontier for real-world complexity. arXiv preprint arXiv:2607.00248, 2026.
  30. [30]Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024.arxiv.org/abs/2402.03300
  31. [31]Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. HuggingGPT: Solving ai tasks with chatgpt and its friends in hugging face. In Advances in Neural Information Processing Systems, pages 38154–38180, 2023.arxiv.org/abs/2303.17580
  32. [32]Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems, pages 8634–8652, 2023.arxiv.org/abs/2303.11366
  33. [33]Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023.
  34. [34]Kimi Team, Tongtong Bai, Yifan Bai, Yiping Bao, SH Cai, Yuan Cao, Y Charles, HS Che, Cheng Chen, Guanduo Chen, et al. Kimi k2. 5: Visual agentic intelligence. arXiv preprint arXiv:2602.02276, 2026.
  35. [35]Suramya Tomar. Converting video formats with ffmpeg. Linux journal, 2006(146):10, 2006.DOI
  36. [36]Bryan Wang, Yuliang Li, Zhaoyang Lv, Haijun Xia, Yan Xu, and Raj Sodhi. Lave: Llm-powered agent assistance and language augmentation for video editing. In Proceedings of the 29th International Conference on Intelligent User Interfaces, pages 699–714, 2024.arxiv.org/abs/2402.10294
  37. [37]Jing Wang, Ao Ma, Ke Cao, Jun Zheng, Zhanjie Zhang, Jiasong Feng, Shanyuan Liu, Yuhang Ma, Bo Cheng, Dawei Leng, Yuhui Yin, and Xiaodan Liang. WISA: World simulator assistant for physics-aware text-to-video generation. arXiv preprint arXiv:2503.08153, 2025.arxiv.org/abs/2503.08153
  38. [38]Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, et al. A survey on large language model based autonomous agents. Frontiers of Computer Science, 18(6):186345, 2024.DOI
  39. [39]Yusong Wu, Ke Chen, Tianyu Zhang, Yuchen Hui, Taylor Berg-Kirkpatrick, and Shlomo Dubnov. Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE, 2023.
  40. [40]An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025.arxiv.org/abs/2505.09388
  41. [41]Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. ReAct: Synergizing reasoning and acting in language models. In International Conference on Learning Representations, 2023.arxiv.org/abs/2210.03629
  42. [42]Yongsheng Yu, Ziyun Zeng, Zhiyuan Xiao, Zhenghong Zhou, Hang Hua, Wei Xiong, and Jiebo Luo. Aurora: Unified video editing with a tool-using agent. arXiv preprint arXiv:2605.18748, 2026.arxiv.org/abs/2605.18748
  43. [43]Zhanjie Zhang, Ao Ma, Ke Cao, Jing Wang, Shanyuan Liu, Yuhang Ma, Bo Cheng, Dawei Leng, and Yuhui Yin. U-StyDiT: Ultra-high quality artistic style transfer using diffusion transformers. arXiv preprint arXiv:2503.08157, 2025.arxiv.org/abs/2503.08157
  44. [44]Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. ExpeL: Llm agents are experiential learners. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 19632–19642, 2024.arxiv.org/abs/2308.10144
  45. [45]Shifang Zhao, Yihan Hu, Ying Shan, Yunchao Wei, and Xiaodong Cun. CutClaw: Agentic hours-long video editing via music synchronization. arXiv preprint arXiv:2603.29664, 2026.arxiv.org/abs/2603.29664
  46. [46]Milton Zhou, Sizhong Qin, Yongzhi Li, Quan Chen, and Peng Jiang. AutoCut: End-to-end advertisement video editing based on multimodal discretization and controllable generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 37777–37787, 2026.arxiv.org/abs/2603.28366

Paper details

vlnc:W_qs4vdf5d
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents