betaFeedback

WorksComputer Science

When Optimization Meets Human Intent: Human-Centered Diffusion Model Optimization

Abstract

Diffusion models are evolving from general tools into personalized systems that adapt to user needs through fine-tuning and guidance. However, diffusion optimization mainly targets measurable objectives such as aesthetics, rarely examining whether they align with human intent. Human needs are multidimensional, contextual, and dynamic, whereas existing methods approximate them with fixed metrics. Because these metrics capture human needs only partially and may amplify deviations from true intent during optimization, higher scores need not yield results better matching human expectations. To understand how human needs enter and influence optimization, we systematically analyze how human intent is expressed, translated into optimizable objectives, applied to model behavior or generation, and evaluated through outputs. We further discuss trade-offs affecting human-intent alignment in personalized image generation and gaps between automated metrics and real user judgments. This human-centered synthesis reorganizes diffusion optimization and provides a systematic reference for personalized generation systems designed for human preferences.

Framework illustrating human-centered diffusion model optimization

Figure 1. The framework illustrates how human intent originates from users and creators, researchers and developers, and domain experts; is represented and translated into machine-optimizable objectives; and subsequently influences diffusion model behavior through model-level and generation-level optimization. It further highlights the key trade-offs arising during optimization, including proxy fidelity, diversity–alignment balance, human agency, and system cost. Finally, it summarizes complementary evaluation practices, ranging from automated metrics to human-grounded evaluations, for assessing whether optimization outcomes genuinely align with evolving human intent..

Introduction

Research on diffusion models [13, 16, 21, 43, 52, 60, 96, 99, 110, 134, 160] is shifting from “how to generate images” to “how to better meet specific human preferences” [71, 130]. In earlier text-to-image paradigms, users expressed intent mainly through prompts, and models generated images accordingly. Recently, fine-tuning [8] and training-free guidance [145] have changed this interaction: generative models no longer merely respond to text prompts, but can adapt their behavior based on human preferences and feedback to optimize toward specific goals [7, 117].

Despite these advances, optimizing generative models toward human preferences does not necessarily ensure that their outputs faithfully reflect human intent [19]. Human needs are often multidimensional, subjective, and context-dependent, making it difficult to directly translate them into computational objectives and verify whether the resulting images truly satisfy user expectations [161]. Consequently, bridging the gap between human intent and personalized image generation involves three interconnected challenges.

(1) User intent must be expressed and translated into computable objectives. User intent must first be externalized through text, choices, ratings, or other forms of feedback, and subsequently translated into machine-optimizable objectives [103, 140]. Thus, what optimization algorithms actually process is not human intent itself, but a proxy representation constructed from these interaction signals [111, 158]. Different forms of expression capture different aspects of user needs, and the same subjective need may be mapped to different computational objectives, potentially resulting in information loss before intent enters the optimization process [23, 80].

(2) Different user needs must be coordinated as optimization objectives. Personalized generation often involves multiple dimensions, such as aesthetic quality, semantic consistency, realism, and diversity [23, 161]. These dimensions may naturally compete with each other, and their importance can change across tasks and generated results [81, 169]. However, existing methods usually combine multiple evaluation objectives in a static way, compressing users’ multidimensional and dynamic needs into optimization signals with fixed weights [58, 59]. Such preset objective relations are hard to adjust with changing user preferences [14]. As a result, the model may keep reinforcing some objectives while sacrificing others that are more important in the current context, causing the optimization direction to deviate from users’ actual preferences [2, 36, 46].

(3) Optimization outcomes must be validated against actual human preferences. Improving predefined optimization objectives does not necessarily mean that generated results better satisfy human needs or align with user intent [45, 116]. Existing studies often evaluate generated outputs using reward scores, text-image alignment, aesthetic metrics, or other automated evaluators, which may share similar assumptions with the objectives used during optimization [61, 128]. Consequently, even when both optimization targets and evaluation metrics improve, it remains unclear whether these improvements correspond to what users actually perceive, prefer, or expect from the generated results [79, 88]. From a human-centered perspective, evaluating personalized generation requires moving beyond automated metrics to consider human judgments, individual preferences, and the specific contexts in which users interact with generative systems [123, 126]. For highly subjective and context-dependent needs, the key is therefore not merely to confirm improvements in proxy metrics, but to examine whether these gains translate into greater satisfaction of human preferences and better alignment with users’ actual intentions [12].

Together, these three challenges highlight that human intent is not directly mapped to the final output of a generative system; instead, it undergoes a continuous transformation process. Human intent must be represented as machine-optimizable objectives, which are then coordinated and translated into changes in model behavior or generation trajectories [25, 102]. Throughout this process, incomplete proxy representations, competing objectives, and optimization mechanisms may introduce tensions and unintended outcomes, potentially weakening alignment with human preferences. Ultimately, whether optimization truly succeeds must be assessed through human-centered validation rather than improvements in automated metrics alone [44, 114]. These interconnected challenges suggest that understanding human-centered diffusion model optimization requires examining not only how optimization is performed, but also how human intent is represented, transformed, potentially distorted, and ultimately evaluated. Therefore, rather than organizing existing studies by algorithm type [11, 51]. This paper re-examines diffusion model optimization through four interconnected dimensions: intent representation, optimization mechanisms, failure modes and trade-offs, and human-centered validation. Based on this perspective, we address the following four research questions:

  • RQ1: How is human intent represented and operationalized in diffusion model optimization?

  • RQ2: How do existing optimization paradigms translate human intent into changes in model behavior or generation trajectories?

  • RQ3: What tensions and failure modes emerge when optimizing diffusion models toward human-defined or human-proxied objectives?

  • RQ4: How is alignment with human intent validated through automated and human-centered evaluations, and what limitations remain in current evaluation practices?

To address these research questions, we systematically review existing studies on diffusion model optimization through a human-centered analytical framework encompassing four interconnected dimensions: intent representation, optimization mechanisms, failure modes and trade-offs, and human-centered validation. Specifically, we investigate how human intent is expressed and translated into machine-optimizable objectives (RQ1), how different optimization paradigms translate these objectives into changes in model behavior or generation trajectories (RQ2), what tensions and failure modes emerge during optimization (RQ3), and how existing evaluation practices assess whether optimization outcomes genuinely align with human preferences and user needs (RQ4). By connecting these dimensions, our review provides a unified perspective on how human intent is represented, operationalized, executed, and validated throughout diffusion model optimization. The main contributions of this paper are as follows:

  • We systematically examine how human intent is expressed, represented, and translated into machine-optimizable objectives, revealing potential information loss and mismatches between user needs and computational proxies.

  • We organize reinforcement-learning fine-tuning and inference-time optimization into model-level and generation-level paradigms, systematically comparing how they translate human-centered objectives into changes in model behavior or generation trajectories.

  • We identify key trade-offs and failure modes in human-centered diffusion model optimization, examining how proxy-objective mismatch, competing objectives, reduced diversity, and optimization costs may undermine alignment with human intent.

  • We systematically examine automated and human-centered evaluation practices, highlighting discrepancies between proxy-based metrics and actual human judgments, and identifying limitations in validating whether optimization outcomes genuinely satisfy user needs.

The overall structure of this survey and the logical relationships among its sections are summarized in Figure 2.

Figure 2. Overview of the Survey and Research Roadmap

Related Work

Human Intent in Diffusion Model Optimization

Diffusion models [29, 67, 85, 106, 108, 125, 148, 166] generate images through iterative denoising, allowing external signals to influence intermediate states and offering more control than one-shot conditioning. Early classifier and classifier-free guidance show that changing sampling directions can shape outputs and trade off quality, alignment, and diversity. Recently, such control has moved toward preference-driven personalized generation. Some methods update model parameters to change long-term behavior [26, 146, 147, 164], while others keep parameters fixed and adjust latents or denoising trajectories at inference time. We call them model-level optimization and *generation-level optimization. From an HCI view, the key is not only what reward is used, but how human needs enter the system [149]. In generative AI, user goals are often clarified through repeated generation, observation, and revision [2, 9, 126]. Prompts are only one expression form and are limited for fine-grained or contextual needs; choices, clicks, and context can further convey intent [23, 103]. Thus, human intent refers to the user’s overall goal for both the desired output and the generation process, while interaction signals are its external representations [9]. We define human-centered diffusion model optimization as obtaining goals related to human needs and using them to actively change model behavior or generation trajectories. These needs may come directly from users or through proxy models [6, 7, 97, 140, 144].

Human-Centered Diffusion Model Optimization

Diffusion model optimization has evolved from improving generation capabilities and controllability toward aligning generated outputs with human preferences and intentions. Early research primarily focused on the fundamental principles and generation capabilities of text-to-image diffusion models [159]. Subsequent studies extended these capabilities through controllable generation, introducing different mechanisms for incorporating specific conditions, multiple constraints, and general control signals into the generation process [11, 51]. While these advances have improved the controllability of diffusion models, they primarily address how generation can be technically controlled rather than whether the resulting outputs genuinely satisfy human needs.

More recently, diffusion model optimization has increasingly incorporated human preferences and feedback into the generation process. Research has explored preference modeling, reinforcement learning from human feedback (RLHF), direct preference optimization (DPO), and test-time alignment to steer generative models toward human-defined objectives [71]. Related studies have examined preference alignment in image generation and editing, highlighting the roles of reward modeling, RLHF, and DPO in adapting generation behavior to human preferences [130]. These developments suggest a growing shift from optimizing predefined technical objectives toward incorporating human preferences into model optimization. However, human preferences are often represented through computational proxies, raising questions about whether improvements in these objectives genuinely reflect users’ underlying intentions.

From a human-centered computing perspective, effective optimization requires not only improving predefined objectives but also understanding how human intent is expressed, interpreted, and incorporated into generative systems. Prior HCI research emphasizes that user goals are not necessarily fixed or fully specified in advance; instead, they may be progressively clarified and revised through interaction, feedback, and contextual information [2, 23, 103]. Accordingly, human-centered diffusion model optimization should be understood as a process that connects human intent with computational objectives, uses these objectives to influence model behavior or generation trajectories, and evaluates whether the resulting outputs genuinely satisfy user needs. This perspective moves beyond classifying optimization techniques solely by their technical mechanisms and instead emphasizes how human intent is represented, operationalized, and maintained throughout the optimization process.

Human-Centered Evaluation of Generative AI

Evaluating generative AI has traditionally relied on automated metrics to assess the quality of generated outputs. In image generation, commonly used measures include visual fidelity, text-image alignment, aesthetic quality, and learned preference scores. These metrics provide scalable and efficient evaluation mechanisms, but their improvements do not necessarily correspond to human perceptions or preferences. Prior research has identified discrepancies between automated metrics and human judgments, highlighting the importance of incorporating human evaluation when assessing text-to-image generation [89, 128]. From a human-centered perspective, evaluation should therefore consider not only measurable output quality but also whether generated results satisfy users’ actual expectations.

HCI research has demonstrated that evaluating generative systems requires attention to preferences and interaction contexts. For example, PromptCharm investigates how multimodal prompting and iterative refinement help users generate images that better match their intentions [126]. Similarly, RoomDreaming examines personalized design exploration by considering users’ preferences, interaction efficiency, and satisfaction throughout the creative process [123]. Collaborative fashion-design systems provide HCI examples of generative AI embedded in multi-stage creative workflows. CrossGAI supports cross-device designer collaboration through AI-assisted sketch retrieval, sketch generation, and sketch-to-image synthesis, whereas HAIGEN integrates text-guided inspiration, personalized sketch-material retrieval, sketch refinement, and coloring [20, 49]. Other studies have shown that AI-generated content can influence users’ creative exploration and decision-making, suggesting that improvements in generation capabilities do not necessarily lead to better interaction experiences [116]. These findings indicate that human-centered evaluation extends beyond assessing generated outputs to understanding how users interact with, interpret, and refine generative results.

Meanwhile, the reliability of human evaluation remains a consideration. Human judgments may vary with evaluation criteria, participant backgrounds, and experimental protocols, making evaluation design and reproducibility essential [89]. Accordingly, human-centered evaluation requires combining automated measurements with human judgments and contextual assessments to determine whether generative systems support user needs. This perspective provides an important foundation for examining how human-intent alignment is evaluated in diffusion model optimization.

Existing Surveys and Our Perspective

Existing surveys have examined diffusion models from several complementary perspectives. Early reviews primarily focused on the foundations, architectures, and generation capabilities of text-to-image diffusion models [159]. Subsequent surveys extended this scope to controllable generation, systematically organizing existing methods according to control conditions, implementation mechanisms, and evaluation strategies [11, 51]. More recently, research on diffusion model alignment has received increasing attention. Liu et al. [71] reviewed alignment objectives, preference modeling, optimization techniques, and evaluation benchmarks, while Wu et al. [130] provided a comprehensive analysis of preference alignment in image generation and editing, covering reinforcement learning from human feedback, direct preference optimization, and reward modeling. Together, these surveys establish an important foundation for understanding the technical development of diffusion models and their alignment with human preferences.

Despite their complementary contributions, these reviews differ from our work in their primary analytical focus. Existing surveys generally emphasize model capabilities, control mechanisms, optimization paradigms, or alignment techniques, whereas the transformation of human intent across different stages of optimization receives less integrated treatment. In particular, how user needs are expressed through interaction, translated into computational proxies, coordinated as optimization objectives, and subsequently validated against actual human preferences is not typically examined as a unified process. This distinction is important because successful optimization of a predefined objective does not necessarily imply that the resulting generation faithfully reflects the user’s underlying intent, especially when preferences are subjective, multidimensional, and context-dependent.

To complement existing surveys, our work adopts human intent transformation as its central analytical perspective. Rather than introducing another taxonomy based primarily on model architectures or optimization algorithms, we connect four interconnected dimensions: intent representation, optimization mechanisms, failure modes and trade-offs, and human-centered validation. This perspective enables us to examine how human needs enter diffusion model optimization, how computational objectives influence model behavior or generation trajectories, where deviations from human intent may emerge, and whether optimization outcomes are genuinely supported by human judgments. By integrating these dimensions into a unified analytical framework, our review provides a human-centered understanding of diffusion model optimization that complements existing technical and preference-alignment surveys. Table 2 compares representative surveys on diffusion models, controllable generation, and preference alignment with our human-centered perspective. Unlike prior surveys primarily organized around generation capabilities, control mechanisms, or alignment techniques, our review examines how human intent is represented, transformed into computational objectives, executed through optimization, potentially distorted, and ultimately validated against actual human needs.

SurveyPrimary PerspectiveOrganizing FrameworkCentral Research Focus
Zhang et al. [159]Text-to-image diffusionGenerative foundations, methods, applications, and challengesHow diffusion models generate images from text
Jiang et al. [51]Controllable diffusion generationControl formulations, mechanisms, and evaluationHow diffusion generation can be guided by control signals
Cao et al. [11]Controllable text-to-image generationSpecific, multiple, and universal control conditionsHow different conditions are incorporated into generation
Liu et al. [71]Human alignment of diffusion modelsRLHF, DPO, test-time alignment, benchmarks, and evaluationHow diffusion models are aligned with human preferences
Wu et al. [130]Preference alignment in image generation and editingReward modeling, RLHF, DPO, applications, and challengesHow preference-alignment techniques improve generation and editing
This ReviewHuman-intent transformationIntent representation → Proxy objectives → Optimization → Trade-offs → Human-centered validationHow human intent is represented, transformed, potentially distorted, and validated throughout optimization

Table 2. Comparison of representative surveys on diffusion models, controllable generation, and preference alignment with our human-centered perspective.

Review Methodology

To systematically examine how human intent enters and shapes diffusion model optimization, we conduct a systematic literature review guided by the four research questions. Our methodology consists of three stages: First, we identify and screen relevant studies through cross-database searches, citation tracking, and predefined selection criteria; Second, we extract technical and human-centered information from the selected studies and conduct iterative thematic analysis;

Figure 4. Evolution of human-centered diffusion model optimization. The timeline illustrates six successive research stages: (1). early exploration of diffusion models with limited human-centered objectives; (2). human-guided optimization through fine-tuning and preference learning; (3). inference-time human control and controllable generation; (4). multimodal and multi-objective alignment; (5). interactive and adaptive optimization systems; and (6). emerging unified human-centered generation frameworks. Representative models and methods are included to illustrate the major developments and shifting research priorities across these stages.

Finally, we organize the resulting evidence around four interconnected dimensions: intent representation, optimization mechanisms, failure modes and trade-offs, and human-centered validation. An overview of the systematic review workflow, including study identification, thematic analysis, and the final analytical scope, is presented in Figure 3.

Figure 3. The review consists of three stages: (1). Identification and study selection, including database searches, keyword-based retrieval, citation tracking, screening, and the application of inclusion and exclusion criteria; (2). Data extraction and thematic analysis, which capture technical and human-centered variables and organize the extracted evidence according to four research questions; and (3). Final corpus and analytical scope, which distinguishes model-level from generation-level optimization and structures the subsequent synthesis around four interconnected dimensions: human intent representation and operationalization (RQ1), optimization mechanisms (RQ2), trade-offs and failure modes (RQ3), and human-centered validation (RQ4).

Search and Study Selection

We searched studies published between January 2020 and September 2026 on diffusion model optimization, human preferences, and human–AI interaction. The databases included ACM Digital Library, IEEE Xplore, Web of Science, Scopus, and arXiv, covering HCI, computer vision, machine learning, and generative AI.

The search strategy was organized around three concept groups, as summarized in Table 3. The search terms were organized into three concept groups. The first focused on diffusion models, including diffusion model, text-to-image diffusion, and diffusion-based image generation. The second identified optimization mechanisms, including reinforcement learning, RLHF, preference optimization, DPO, reward optimization, training-free guidance, inference/test-time optimization, latent optimization, and trajectory optimization. The third focused on human intent and interaction, including human preference, human feedback, human intent, user preference, alignment, personalization, human-centered, user interaction, interactive generation, iterative refinement, and human-AI co-creation. Since different research communities describe human-centered optimization using different terminology, we did not require studies to explicitly mention human intent or human-centered. Instead, we examined whether their optimization objectives could be meaningfully connected to human needs, preferences, or interaction signals during full-text screening.

ACM Digital Library & IEEE Xplore
Title("diffusion model" OR "text-to-image diffusion" OR "diffusion-based image generation")
Text("reinforcement learning" OR "RLHF" OR "DPO" OR "preference optimization" OR "reward optimization" OR "training-free guidance" OR "inference-time optimization" OR "test-time optimization" OR "latent optimization" OR "trajectory optimization") AND ("human preference" OR "human feedback" OR "human intent" OR "user preference" OR "alignment" OR "personalization" OR "human-centered" OR "user interaction" OR "interactive generation" OR "iterative refinement" OR "human-AI co-creation")
Web of Science & Scopus
Title("diffusion model" OR "text-to-image diffusion" OR "diffusion-based image generation")
Topic / Keywords("reinforcement learning" OR "RLHF" OR "DPO" OR "preference optimization" OR "reward optimization" OR "training-free guidance" OR "inference-time optimization" OR "latent optimization" OR "trajectory optimization") AND ("human preference" OR "human feedback" OR "human intent" OR "user preference" OR "alignment" OR "personalization" OR "human-centered" OR "interactive generation" OR "user interaction")
arXiv
Title / Abstract("diffusion model" OR "diffusion-based generation" OR "text-to-image") AND ("preference" OR "feedback" OR "alignment" OR "reinforcement learning" OR "reward" OR "guidance" OR "optimization" OR "personalization")

Table 3. Keyword search terms for identifying studies on human-centered diffusion model optimization.

To reduce omissions caused by keyword variation, we further conducted forward and backward citation tracking on representative studies of reinforcement-learning fine-tuning, preference optimization, training-free guidance, and diffusion alignment, and examined references in related surveys. All studies identified through citation tracking were screened using the same criteria as database results.

The selection process included deduplication, title–abstract screening, and full-text assessment. We excluded surveys, opinion articles, tutorials, posters, theses, and short papers without complete method descriptions from the core research corpus; relevant surveys were retained for background analysis and citation tracking. During full-text screening, studies were assessed according to our definition of human-centered diffusion model optimization. Included studies had to satisfy two criteria. First, diffusion models had to serve as the primary generation mechanism, with the proposed method modifying model parameters or the generation process. Second, the optimization objectives had to be directly or indirectly connected to human needs, preferences, or evaluations. Such connections could be established through explicit user feedback, ratings, rankings, pairwise preferences, or computational proxies, including reward and preference models trained on human evaluation data [167]. Importantly, these different forms of human involvement were not treated as equivalent, since computational proxies may capture only selected aspects of the original human intent.

Accordingly, we excluded studies focused solely on pretraining, model compression, sampling acceleration, distillation, memory efficiency, or numerical solver efficiency. General conditional or controllable generation was not included simply because it employed textual, positional, or structural conditions, unless these signals further participated in preference-driven optimization. Studies focusing on human–AI interaction, user experience, intent expression, or evaluation without directly optimizing diffusion models were retained as contextual references rather than included in the core corpus. Similarly, works proposing reward models, evaluation metrics, or benchmarks without optimizing diffusion models were used as supplementary evidence for analyzing human-centered evaluation practices.

Data Extraction and Thematic Analysis

For each included study, we extracted information related to human intent, including intent source, expression form, proxy construction, user involvement, and the stage at which human preferences influenced optimization. We also recorded technical characteristics, including publication year, generation task, base model, optimization objectives, intervention mechanisms, and evaluation methods. These variables allowed us to examine both how human needs were operationalized and how the resulting objectives shaped diffusion model optimization.

The coding framework was organized around the four research questions. For RQ1, we examined how human intent was expressed, represented, and translated into rewards, preference signals, or other computational proxies. For RQ2, we analyzed how these objectives influenced model behavior or generation trajectories, distinguishing model-level and generation-level optimization. For RQ3, we recorded proxy-objective mismatches, competing objectives, diversity-related trade-offs, optimization costs, and reported failure modes. For RQ4, we extracted automated metrics, human judgments, expert evaluations, and contextual validation methods, with particular attention to whether evaluation practices reflected individual preferences and actual user needs.

We then conducted iterative thematic analysis to identify recurring patterns and differences across studies. Rather than organizing the literature solely by algorithm type, we compared studies according to how human intent was represented, how optimization objectives were applied, what tensions emerged, and how alignment outcomes were evaluated. Related codes were grouped into broader themes, while distinctions in intent sources, proxy construction, and optimization intervention points were preserved. Ambiguous cases were revisited to refine the coding categories and maintain consistent interpretation across the research corpus.

Final Corpus and Analytical Scope

The final research corpus covered two main pathways through which human-centered objectives influence diffusion model optimization. Model-level optimization updates model parameters to incorporate human preferences into generation behavior, enabling relatively persistent adaptation across subsequent generations. In contrast, generation-level optimization keeps the base model fixed and adjusts the current generation process through guidance, latent optimization, or trajectory intervention. These pathways differ not only in their optimization mechanisms but also in how human intent influences generation behavior and when users can intervene. Since some studies involved multiple mechanisms, we categorized them according to their primary intervention points and the ways in which optimization objectives were applied, without treating the two pathways as mutually exclusive.

Rather than organizing the corpus solely by algorithm type, we structured the analysis around the four research questions. Specifically, we first examine how human intent is expressed, represented, and operationalized into computational objectives (RQ1). We then analyze how these objectives influence model behavior or generation trajectories through different optimization mechanisms (RQ2). Next, we investigate the trade-offs, tensions, and failure modes that may emerge when optimizing toward human-defined or human-proxied objectives (RQ3). Finally, we examine how automated and human-centered evaluation practices validate whether optimization outcomes genuinely align with human preferences and user needs (RQ4). Together, these dimensions provide a unified analytical scope for understanding how human intent is represented, executed, potentially distorted, and evaluated throughout diffusion model optimization. To provide an overview of the analytical structure developed in the following sections, Table 1 summarizes the proposed taxonomy tables across Sections 4–9. It outlines the main classification dimensions covered in each section, ranging from the stakeholders shaping optimization goals and the operationalization of human intent to optimization methods, associated trade-offs, evaluation practices, and future research directions. This overview provides a roadmap for the subsequent systematic analysis.

SectionProposed Table TitleMain Classification Dimensions
§ 4Stakeholders and Their Roles in Shaping Optimization GoalsEnd users, researchers, domain experts, and computational proxies.
§ 5Taxonomy of Human Intent OperationalizationTransformation of human intent into constraints, reward signals, preference models, and multi-objective optimization signals.
§ 6Taxonomy of Human-Centered Diffusion Optimization MethodsReinforcement learning fine-tuning, direct preference optimization (DPO), training-free guidance, and latent/trajectory optimization.
§ 7Taxonomy of Optimization Trade-offs and Failure ModesReward hacking, diversity loss, human agency, and computational and interaction costs.
Sec. 8Taxonomy of Evaluation Practices for Human-Intent AlignmentAutomated metrics, preference-based evaluation, human judgments, expert evaluation, and interactive and longitudinal evaluation.
§ 9Research Gaps and Future DirectionsAdaptive intent modeling, dynamic objective coordination, user control, and interactive evaluation.

Table 1. Proposed Taxonomy Tables Across the Survey Chapters

The Landscape of Human-Centered Diffusion Model Optimization

This section characterizes the research landscape of human-centered diffusion model optimization through three interconnected dimensions derived from our thematic analysis. First, we examine the diverse human goals that existing optimization methods seek to satisfy, including generation quality, task fidelity, and individual preferences. Second, we analyze who defines and shapes these optimization goals, considering the roles of users, researchers, domain experts, and computational proxies. Third, we investigate how and at which stages human intent enters the optimization process through different interaction signals. Together, these dimensions provide a human-centered understanding of the research landscape and establish the foundation for the next section, which further examines how human intent is represented and translated into machine-optimizable objectives. Figure 5 summarizes these three dimensions, providing an integrated taxonomy of human goals, the stakeholders shaping optimization objectives, and the entry points through which human intent influences the optimization process.

Figure 5. A human-centered taxonomy of human intent in diffusion model optimization, organized along three dimensions: (1) What: the goals that human-centered optimization seeks to satisfy, including output quality, task fidelity, individual preferences, and process control; (2) Who: the stakeholders who define or mediate optimization goals, including end users, researchers, domain experts, and computational proxies; and (3) Where: the entry points through which human intent influences optimization, including objective construction, model adaptation, inference-time intervention, and iterative feedback. Together, these dimensions characterize how human intent is expressed, shaped, operationalized, and continuously refined throughout diffusion model optimization.

What Human Goals Are Targeted in Diffusion Optimization?

Existing studies often describe their goals as ‘better meeting human needs’, but these needs encompass different dimensions of human intent. Some methods primarily focus on the quality of generated outputs, such as aesthetic appeal, visual realism, or perceptual quality. Others emphasize how faithfully the model responds to user requests, including text-image alignment and the accurate depiction of objects, attributes, and relations. Thus, even when described as alignment, different methods may prioritize aesthetic preferences or task fidelity [40, 63].

With the development of personalized generation, optimization goals have increasingly expanded beyond group-level preferences to account for individual differences. The central question is therefore not only ‘what kind of image is generally better’, but also ‘what kind of result better satisfies the needs of a particular user’. Some studies incorporate personal choices, historical feedback, or repeated interactions to adapt generation behavior to individual preferences [66, 93, 133, 141, 165].

Human needs extend beyond the quality of the final image. In interactive creation and editing, users may value structure preservation, local controllability, output diversity, and opportunities for creative exploration. These aspects influence not only what users expect from generated results but also how they wish to interact with and control the generation process. As summarized in Table 4, human intent in diffusion model optimization encompasses four categories: output quality, task fidelity, individual preferences, and process-level control. These categories are not mutually exclusive; they may coexist within a task and evolve in relative importance as users interact with the system.

Intent categoryCore concernTypical optimization goalsRepresentative user needs / signals
Quality-OrientedImproving the perceived quality of generated resultsAesthetic appeal, visual realism, perceptual quality, image attractiveness“Make it look better,” “more realistic,” “more beautiful,” or preferences reflected in ratings or pairwise judgments
Fidelity-OrientedEnsuring that outputs faithfully satisfy the user’s request or task specificationText–image alignment, accurate depiction of objects, attributes, relations, and structural constraints“Follow my prompt more accurately,” “preserve the object layout,” “edit only this part,” or instruction-following feedback
Preference-OrientedAdapting generation to differences across users rather than optimizing only group-level averagesPersonalized style preference, user-specific aesthetics, historical preference adaptation, identity-consistent generationPersonal choices, historical interactions, repeated feedback, user profiles, or multi-round preference refinement
Control-OrientedSupporting how users interact with and steer the generation process, not only the final resultStructure preservation, local controllability, output diversity, exploration support, editability, interactive refinementRegion selection, reference images, iterative edits, requests for multiple alternatives, and needs for controllable exploration

Table 4. Taxonomy of human goals targeted in diffusion model optimization. These categories are analytically distinguishable but often co-exist within a single generation or editing task.

Who Shapes the Optimization Goal?

Optimization goals in diffusion models are not always directly defined by end users. In user-driven settings, individuals express their needs through text, ratings, rankings, or preference comparisons, allowing their judgments to influence the optimization direction. However, the extent of this influence depends on how these signals are interpreted and incorporated into the optimization process.

In many studies, researchers determine which attributes should be optimized, such as aesthetic quality, text-image alignment, or task-specific performance. Human feedback is then collected or interpreted within these predefined dimensions. Consequently, even when optimization relies on human evaluations, the underlying definition of what constitutes a better result may still be largely determined by the research design. Similarly, in professional domains such as medicine or design, domain experts and established professional standards may influence the definition of desirable outputs. In these settings, optimization goals can reflect both individual user needs and domain-specific requirements.

Computational proxies introduce a layer of mediation between human judgments and optimization objectives. Reward models, preference models, and VLM-based evaluators can transform human evaluations or predefined criteria into automatic optimization signals. However, these proxies may preserve certain aspects of human preferences while overlooking others, thereby influencing which objectives are reinforced during optimization [41]. Thus, optimization goals may be shaped through the combined influence of users, researchers, domain experts, and computational proxies, although their roles in defining and mediating these goals differ. Distinguishing these roles is essential for understanding whose preferences are represented and how faithfully the objectives reflect the intentions of specific users.

Where Does Human Intent Enter the Optimization Process?

Human intent can be expressed through various interaction signals, including text prompts, ratings, rankings, pairwise preferences, region selection, reference images, multi-turn edits, interaction history, and expert judgments in professional settings. These signals differ in their information granularity, user effort, and ability to communicate specific preferences or constraints. However, not every interaction signal directly participates in optimization. Within the scope of this review, such signals become relevant to diffusion model optimization when they are used to construct objectives, guide model adaptation, or actively intervene in the generation process.

Human intent can influence optimization at different stages rather than entering the system only through an initial request. Some studies incorporate human preferences during feedback collection and objective construction, translating user judgments into rewards, preference models, or other computational proxies. Others introduce these objectives during training or fine-tuning to influence subsequent generation behavior. Alternatively, human-defined or human-proxied objectives can guide the current generation process at inference time through latent adjustments, guidance mechanisms, or trajectory interventions [1, 22, 42]. After generation, users may further provide choices, ratings, or editing instructions that inform subsequent rounds of interaction and optimization. Figure 6 provides an overview of the interaction signals through which human intent is expressed and illustrates how these signals influence different stages of the diffusion model optimization process, from objective construction and model adaptation to inference-time intervention and iterative user feedback.

Figure 6. Human intent and its entry points in diffusion model optimization. Human needs are expressed through diverse interaction signals and can influence objective construction, model adaptation, inference-time intervention, and iterative feedback. These entry points are not mutually exclusive and may jointly support evolving user preferences across generation and editing tasks.

Therefore, human intent should not be understood as a fixed condition supplied only at the system input. Instead, it may enter and influence diffusion model optimization through objective construction, model adaptation, inference-time intervention, and iterative user feedback. Identifying these entry points helps clarify when users can influence generation behavior and provides a basis for examining how their expressed intentions are subsequently transformed into machine-optimizable objectives.

From Human Intent to Optimization Objectives

§ 4 has characterized what human goals existing studies seek to satisfy, who shapes these goals, and where human intent enters diffusion model optimization. Building on this analysis, this section examines how expressed human needs are represented and operationalized into computational objectives, thereby addressing RQ1. Specifically, we trace the transformation from intent representation and proxy construction to objective coordination and potential proxy mismatches, examining how human intent becomes machine-optimizable and where information may be lost or distorted. Table 5 summarizes the main categories of human intent representation and operationalization identified in our review. Specifically, we distinguish intent representation, proxy objective construction, objective coordination, and proxy mismatch, highlighting how human needs are progressively transformed into machine-optimizable objectives and where discrepancies may arise. This classification provides an analytical overview of RQ1 and establishes the foundation for the following discussion.

CategorySub-topicDescriptionReferences
Intent RepresentationExplicit Requirements and ConstraintsFormalizes expressed needs as semantic attributes, structural constraints, or task conditions.[139]
Intent RepresentationMultimodal and Personalized RepresentationsUses multimodal information, interaction histories, or personalized representations to encode user needs.[19, 65, 75]
Proxy Objective ConstructionReward and Preference ModelingConverts human judgments into reusable reward functions or preference-based objectives.[18, 38, 59, 64, 68, 76, 122, 124, 138, 155, 156]
Proxy Objective ConstructionPredefined Computational EvaluatorsEmploys predefined evaluation criteria or learned computational representations as optimization proxies.[154]
Objective CoordinationMulti-Objective AggregationCoordinates potentially competing objectives involving visual quality, fidelity, diversity, and user preferences.[135, 143, 146]
Proxy MismatchHuman–Proxy GapCharacterizes potential information loss or misalignment between human needs and their computational representations.[54, 73, 152]

Table 5. Taxonomy of Human Intent Representation and Operationalization. Existing studies and analytical concepts are organized according to how human needs are represented, transformed into computational objectives, coordinated, and potentially misaligned during optimization.

From Expressed Intent to Machine-Readable Representations

Human needs, particularly those that are subjective or context-dependent, cannot always be directly used as computational optimization objectives. Once expressed through interaction signals, human intent must be formalized into attributes, constraints, or preferences that can be represented, compared, and computed. For example, ‘better matching my taste’ may be represented through aesthetic quality, visual realism, or style preferences, while ‘better following my request’ may be expressed through semantic consistency, object attributes, or structural constraints [139].

This formalization determines what information subsequent optimization mechanisms can access. Explicit requirements can often be preserved as conditions or target attributes, whereas subjective preferences may require ratings, rankings, or pairwise comparisons to become computationally interpretable. More complex needs may also be represented through multimodal inputs, interaction history, or personalized user models [19, 65, 75].

Therefore, intent representation is not simply the preservation of user input, but a selective transformation of human needs into machine-readable information. While this process enables computational optimization, it also determines which aspects of the original intent are retained and which may be omitted before subsequent optimization begins.

From Human Signals to Proxy Objectives

Machine-readable representations of human intent do not necessarily constitute objectives that optimization algorithms can directly optimize. Whether these representations originate from textual instructions, preference judgments, or other interaction signals, they often need to be translated into computational objectives that can evaluate generated results and provide optimization signals. This translation forms a critical connection between human intent expression and diffusion model optimization.

One common approach constructs reward or preference models from human evaluations, allowing discrete judgments to be approximated by reusable functions [18, 38, 59, 64, 68, 76, 122, 124, 138, 155, 156]. Approaches employ predefined aesthetic metrics, vision-language models, or task-specific evaluators to approximate selected aspects of human needs as scores. Although these objectives differ in their sources and construction methods, they serve a similar function in optimization: providing measurable signals that guide model updates or generation-time interventions.

This operationalization improves the computability and scalability of human-centered objectives, but it also introduces an additional layer between human intent and algorithmic optimization. Rather than directly optimizing the original human need, algorithms operate on proxy objectives derived from human judgments, learned representations, or predefined evaluation criteria [154]. Consequently, proxy construction determines not only which objectives can be optimized, but also which aspects of human intent are emphasized or overlooked during subsequent generation.

Multi-Objective Trade-offs and Evolving Intent

After human needs are translated into computational objectives, a further challenge arises from the relationships among these objectives. Personalized generation often involves multiple dimensions, including aesthetic quality, semantic consistency, realism, diversity, and structure preservation. However, these objectives cannot always be improved simultaneously. For example, stronger semantic constraints may restrict the available exploration space, while emphasizing aesthetic rewards may favor a narrower range of high-scoring outputs [135, 143, 146]. Optimization systems often employ weighting, ranking, or other aggregation strategies to coordinate competing objectives.

A central difficulty is that objective relationships are frequently established through relatively static weights or predefined priorities, whereas human preferences may evolve across tasks and interactions. For instance, users may initially prioritize semantic fidelity but later place greater importance on diversity after observing several generated candidates. Similarly, during image editing, structure preservation may become increasingly important as an output approaches the desired result. Such changes mean that a fixed combination of objectives may no longer accurately reflect the user’s current priorities.

Therefore, multi-objective coordination is not merely a technical problem of optimizing several metrics simultaneously. It also concerns whether the relationships among computational objectives remain consistent with the user’s evolving intent. The way objectives are combined effectively determines what the optimization system considers a better result. When these relationships fail to adapt to changing human preferences, the constructed optimization objective may gradually diverge from what the user actually seeks to achieve.

The Human–Proxy Gap

Based on the preceding analysis, we characterize the potential structural mismatch between human intent and the computational objectives used by optimization algorithms as the Human–Proxy Gap. Before human needs can guide optimization, they undergo several transformations, including intent expression, machine-readable representation, proxy construction, and objective coordination. Each transformation may preserve certain aspects of human intent while omitting contextual information, individual differences, or relationships among objectives [73]. Consequently, the resulting proxy objectives may represent only part of what users actually want.

Importantly, the Human–Proxy Gap is not equivalent to reward prediction error [54]. Even when a reward model accurately predicts its predefined evaluation target, the optimization system may still produce unsatisfactory results if that target does not fully capture the user’s needs, if competing objectives are combined inappropriately, or if preferences change over time. In such cases, the proxy objective may be accurately estimated and successfully optimized while remaining misaligned with the user’s current intent [152].

From this perspective, addressing RQ1 requires more than identifying which rewards or preference signals are used in existing methods. It requires understanding how human needs are progressively transformed into computational objectives and where discrepancies may emerge during this process. The Human–Proxy Gap therefore provides a foundation for the analysis of optimization mechanisms. The next section examines how model-level and generation-level optimization execute these constructed objectives to influence model behavior or generation trajectories.

Optimizing Diffusion Models Toward Human Intent

This section addresses RQ2 introduced in § 1. § 5 has examined how human needs are represented and transformed into machine-optimizable proxy objectives. Building on this analysis, we investigate how these objectives are applied to diffusion models and translated into changes in generation behavior. Existing methods can be broadly organized into two pathways: model-level optimization, which updates model parameters to influence subsequent generations, and generation-level optimization, which keeps the base model fixed while intervening in the current generation process. Beyond comparing their technical mechanisms, we examine how these pathways differ in the persistence of preference adaptation and the opportunities they provide for user intervention, thereby connecting diffusion model optimization with human agency. Table 6 summarizes the main categories of human-centered diffusion model optimization methods, distinguishing model-level and generation-level approaches according to their optimization mechanisms and intervention points. To provide a consolidated view of the two analytical dimensions, Figure 7 presents complementary taxonomies developed in § 5 and § 6. The left panel organizes human intent operationalization into four dimensions: intent representation, proxy objective construction, objective coordination, and proxy mismatch. It further identifies representative mechanisms within each dimension, including explicit requirements, personalized representations, reward modeling, computational evaluators, and multi-objective aggregation. The right panel categorizes human-centered diffusion optimization methods into two primary paradigms: model-level optimization, which includes reinforcement learning fine-tuning and preference-based optimization, and generation-level optimization, which encompasses training-free guidance and latent or trajectory optimization. Together, these taxonomies connect the construction of machine-optimizable objectives with their subsequent execution in diffusion models, providing a unified view of how human intent is translated into and acted upon by optimization mechanisms.

Taxonomies of human-intent operationalization and human-centered diffusion model optimization

Figure 7. (a) Chapter 5: From Human Intent to Optimization Objectives. (b) Chapter 6: Optimizing Diffusion Models Toward Human Intent. Taxonomies of human-intent operationalization and human-centered diffusion model optimization. (a) From Human Intent to Optimization Objectives: A taxonomy of how human needs are represented and translated into computational objectives, covering intent representation, proxy objective construction, objective coordination, and human–proxy mismatch. (b) From Objectives to Generation Processes: A taxonomy of human-centered diffusion optimization methods, distinguishing model-level optimization, including reinforcement learning fine-tuning and preference-based optimization, from generation-level optimization, including training-free guidance and latent/trajectory optimization. Together, the two taxonomies illustrate how human needs are transformed into computational objectives and subsequently operationalized through different diffusion optimization mechanisms.

CategorySub-topicDescriptionExamples
Model-Level OptimizationReinforcement Learning Fine-TuningUses reward signals and policy updates to modify model parameters toward preferred generation behavior.[37, 48, 82, 142]
Model-Level OptimizationPreference-Based OptimizationLearns from preferred and rejected outputs, often without an explicitly trained scalar reward model.[3, 35, 39, 69, 74, 87, 112, 120, 131]
Generation-Level OptimizationTraining-Free GuidanceModifies denoising or sampling directions using external objectives without updating base-model parameters.[47, 56, 92, 119]
Generation-Level OptimizationLatent and Trajectory OptimizationUpdates latent representations, explores candidate trajectories, or selects generation paths during inference.[5, 31, 50, 55, 105, 115, 127, 129, 168]

Table 6. Taxonomy of Human-Centered Diffusion Model Optimization Methods. Existing approaches are categorized according to their primary intervention level and optimization mechanism.

Model-Level: Persistent Intent Adaptation

Model-level optimization updates model parameters according to constructed proxy objectives, allowing human preferences to influence subsequent generation behavior beyond a single inference process. Representative approaches include reinforcement learning fine-tuning and preference-based optimization. Although these approaches differ in how preference information is represented and incorporated into training, both aim to adjust the model’s generation distribution toward outputs favored by their respective objectives.

Reinforcement learning fine-tuning typically evaluates generated outputs using reward signals and updates model parameters to increase the likelihood of high-reward outcomes or generation trajectories. Through this process, human-defined or human-proxied objectives are translated into parameter updates that reshape the model’s generation behavior [37, 48, 82, 142]. Preference-based optimization, in contrast, learns from relative judgments between preferred and less-preferred samples, allowing models to adapt their generation behavior without necessarily relying on an explicitly trained scalar reward model [3, 35, 39, 69, 74, 87, 112, 120, 131]. Despite their differences, both approaches translate preference-related supervision into changes in the likelihood of subsequent generated outputs.

From a human-centered perspective, model-level optimization transforms preferences expressed during training into relatively persistent tendencies in model behavior. This can reduce the need to repeatedly specify the same preferences during subsequent generations. However, adaptation through parameter updates also introduces a distinction between preferences incorporated during training and those expressed by users in a particular interaction. While users may still influence outputs through prompts or other conditioning signals, changing previously learned tendencies may require additional adaptation or sufficiently effective inference-time control [10, 15, 33, 94, 101, 104, 109]. Thus, model-level optimization emphasizes persistent preference adaptation, while the extent to which individual users can subsequently revise these preferences depends on the available interaction and control mechanisms.

Generation-Level: Inference-Time Intent Execution

Generation-level optimization keeps the parameters of the base model fixed and applies human-defined or human-proxied objectives to influence the current generation process. Unlike model-level optimization, which modifies the model’s learned behavior across subsequent generations, generation-level approaches intervene in how a particular output is produced. This allows optimization objectives and constraints to be adapted to the requirements of individual generation tasks without retraining the base model.

One group of methods introduces preference-related objectives into the denoising or sampling process through training-free guidance, modifying generation directions according to external signals [47, 56, 92, 119]. Another group directly intervenes in latent representations or generation trajectories by updating intermediate states, exploring candidate trajectories, or selecting generation paths according to computational objectives [5, 31, 50, 55, 105, 115, 127, 129, 168]. Although these approaches differ in their intervention mechanisms, they share the objective of steering the current generation process without permanently updating the base model parameters.

From the perspective of human intent, generation-level optimization provides a mechanism for adapting optimization objectives to the current task, prompt, or user. Depending on the method, objectives, guidance strengths, or local constraints may be adjusted between generation attempts or during the generation process. This flexibility can support task-specific personalization and iterative refinement. However, inference-time control does not necessarily imply direct user control, since optimization objectives may also be predefined by researchers or determined through computational proxies. Moreover, some methods introduce additional evaluation, gradient computation, or trajectory search during inference, creating different computational requirements and interaction costs [30, 84, 95]. Therefore, generation-level optimization emphasizes immediate, task-specific intent execution, although the degree of user involvement and computational overhead varies across methods.

Human Agency Across Optimization Paradigms

The distinction between model-level and generation-level optimization concerns not only where optimization occurs, but also when and how users can influence the objectives that guide generation. Model-level optimization typically incorporates human preferences during feedback collection, reward construction, or preference-based training. These preferences can subsequently influence multiple generations without requiring repeated user input. However, the resulting model behavior may reflect aggregated or previously collected preferences rather than the evolving intentions of a particular user.

Generation-level optimization, by contrast, provides opportunities to modify optimization objectives and constraints at inference time. This can allow generation behavior to respond more directly to changing task requirements or user preferences without additional model training. Nevertheless, such opportunities depend on whether the system exposes meaningful controls and allows users to revise the objectives used during optimization. When objectives are determined entirely by predefined evaluators, inference-time optimization may remain largely automated despite operating at the generation level.

Consequently, human agency cannot be inferred solely from whether model parameters are updated. A more meaningful distinction concerns when preferences are incorporated, who determines the optimization objectives, and to what extent users can inspect, revise, or override these objectives during interaction. Model-level optimization generally emphasizes persistent preference adaptation, whereas generation-level optimization offers greater potential for immediate objective adjustment. These differences provide a foundation for the next section, which examines how optimization mechanisms may introduce tensions between intent alignment, generative diversity, user control, and system costs.

When Optimization Fails the User: Tensions and Trade-offs

This section addresses RQ3 introduced in § 1. § 6 has examined how model-level and generation-level optimization translate human-defined or human-proxied objectives into changes in model behavior or generation trajectories. Building on this analysis, we investigate the tensions and unintended consequences that may emerge when these objectives are optimized. From a human-centered perspective, improvements in predefined objectives do not necessarily translate into better outcomes for users. We examine four interconnected dimensions: whether optimization amplifies mismatches between proxy objectives and human intent, how stronger alignment may constrain generative diversity and creative exploration, how increasing automation affects human agency, and what computational and interaction costs accompany optimization gains. Table 7 summarizes the major trade-offs and failure modes identified in human-centered diffusion model optimization. Specifically, we distinguish four interconnected dimensions: proxy fidelity, generative openness, human agency, and system cost. These dimensions highlight how improvements in computational objectives may introduce unintended consequences for user preference satisfaction, creative exploration, control over generation, and interaction efficiency. This classification provides an analytical overview of RQ3 and establishes the basis for examining the limitations of existing optimization paradigms.

CategoryFailure Mode / Trade-offHuman-Centered ImplicationSupporting References
Proxy FidelityReward Hacking and OveroptimizationHigher computational scores may not correspond to improved user satisfaction.[163]
Generative OpennessAlignment–Diversity Trade-offStronger optimization may reduce output diversity and creative exploration opportunities.[137, 157]
Human AgencyAutomated Objective ControlUsers may have limited ability to inspect, revise, or override objectives determined by the system.§ 6.3 synthesis
System CostOptimization Gain vs. ComputationAdditional training or inference computation may reduce interaction efficiency.[30, 84]

Table 7. Taxonomy of Trade-offs and Failure Modes in Human-Centered Diffusion Optimization. The table summarizes major tensions between computational optimization and human-centered outcomes, including proxy fidelity, generative openness, human agency, and system costs.

Proxy Fidelity under Optimization

A central failure mode arises when optimization amplifies discrepancies between computational proxy objectives and human intent. As discussed in § 5, rewards, preference scores, and other automated evaluation signals may capture only selected aspects of human needs. When optimization repeatedly reinforces these proxies, the model may increasingly favor generation patterns that achieve higher computational scores without necessarily producing results that better satisfy users’ actual intentions.

One manifestation of this problem is reward hacking or reward overoptimization. Although an evaluator may approximate human judgments well on ordinary samples, sustained optimization can shift generated outputs toward regions where the evaluator’s predictions are less reliable. Consequently, reward scores may increase while undesirable effects, such as excessive stylization, visual artifacts, or concentration around high-scoring patterns, emerge [163]. Such outcomes illustrate how optimization can amplify limitations already present in proxy objectives, potentially increasing the discrepancy between computational success and human preference satisfaction.

From a human-centered perspective, the central concern is therefore not simply whether an optimization algorithm can maximize its objective, but whether the resulting improvement remains meaningful to users. Optimization success and human-intent alignment should be distinguished: the former concerns how effectively a computational objective is optimized, whereas the latter concerns whether the resulting outputs continue to satisfy users’ actual needs and preferences. Thus, maintaining proxy fidelity during optimization is essential for preventing computational improvements from reinforcing unintended generation behaviors.

Alignment vs. Generative Openness

Even when proxy objectives reasonably reflect human preferences, stronger optimization may still restrict the range of outputs available to users. Concentrating generation toward high-reward regions can improve consistency with predefined objectives, but it may also reduce the diversity of generated results by favoring a narrower set of patterns. This creates a potential trade-off between alignment strength and generative diversity [137, 157].

From a human-centered perspective, this trade-off extends beyond statistical diversity to the opportunities users have for creative exploration. In open-ended generation tasks, users may not have fully formed preferences or a clearly defined final result before interacting with the system. Instead, their intentions can evolve as they observe, compare, and refine generated candidates. When optimization strongly favors a limited range of high-scoring outputs, users may receive results that consistently satisfy predefined objectives while encountering fewer alternative compositions, styles, or unexpected possibilities. This can restrict opportunities to discover new preferences or revise their original intentions during the creative process.

A related tension arises between controllability and creative freedom. More strongly enforcing textual, structural, or local constraints may improve the predictability and fidelity of generated outputs, but can also limit opportunities for exploration beyond explicitly specified requirements. However, such trade-offs are context-dependent: users performing precise editing tasks may prioritize controllability, whereas those engaged in open-ended creation may value diverse alternatives and unexpected results. Therefore, human-centered optimization should not pursue maximum alignment or control as universal objectives. Instead, it should consider how much generative freedom to preserve according to the task, the user’s current preferences, and the extent to which their intent remains open to exploration.

Optimization Gain vs. System Cost

Stronger personalized optimization also usually comes with additional system costs. Model-level approaches require reinforcement learning, preference optimization, or other extra training processes, while generation-level approaches may repeatedly call evaluators, compute gradients, search latent states, or compare multiple generation trajectories during inference. Thus, although different methods place computational overhead at different stages, they all face a trade-off between optimization gain and computational cost [78].

This issue is especially important in human-computer interaction settings, because computational overhead eventually appears as waiting time, response speed, and the number of possible exploration attempts. For tasks that require repeated generation, comparison, and revision, obtaining a higher reward for a single output does not necessarily mean that the whole interaction process becomes more effective. If each optimization step significantly increases response time, users may instead be able to try fewer candidates.

Therefore, human-centered evaluation should not only compare the highest optimization performance a method can achieve, but also consider how much training resource, inference time, and interaction delay are required to obtain such improvement. A more meaningful question is whether the improvement brought by additional optimization effort is sufficient to compensate for its system cost [32], and whether the system can dynamically decide when stronger optimization is worth using according to the task and user needs.

Evaluating Alignment: Are We Measuring What Users Actually Want?

This section addresses RQ4 introduced in § 1, examining how existing studies validate alignment with human intent and what limitations remain in current evaluation practices. The preceding sections have analyzed how human intent is represented, transformed into computational objectives, and executed through different optimization mechanisms, as well as the tensions that may arise during this process. Building on these analyses, we examine how optimization outcomes are evaluated and whether the resulting evidence reflects users’ actual needs. Specifically, we distinguish automated evaluation from human-grounded validation, investigate potential disagreements between computational proxies and human judgments, and discuss the limitations of static evaluations in capturing evolving preferences, interactive experiences, and long-term use.

Automated Evaluation and Proxy-Based Evidence

Automated metrics and reward models are widely used to evaluate diffusion model optimization. Text-image alignment, aesthetic quality, visual realism, and task-specific metrics provide scalable measurements of generated outputs, while learned reward models approximate selected aspects of human preference through evaluation data. These tools support efficient and standardized comparisons across generation methods and optimization settings.

However, automated evaluators capture different and often limited dimensions of human intent. Text-image alignment primarily measures semantic correspondence, aesthetic models emphasize visual appeal, and learned preference evaluators reflect the data distributions and judgment criteria used during their development. Consequently, improvements across multiple automated metrics demonstrate progress on selected computational criteria, but do not necessarily establish that generated outputs satisfy the full range of user needs [28, 34, 53, 57, 77, 86, 100, 113, 121, 136, 151, 153].

A further limitation arises when optimization objectives and evaluation metrics are closely related. If a model is optimized using a particular reward function and subsequently evaluated primarily through the same or a closely related evaluator, higher scores provide evidence that the model better satisfies that computational objective, but not necessarily independent evidence of improved human-intent alignment. From a human-centered perspective, automated metrics therefore provide valuable evidence of measurable performance, while their ability to validate actual preference satisfaction depends on how well their evaluation criteria correspond to the intended human goals.

Human-Grounded and Contextual Validation

Human-grounded evaluation complements automated metrics by incorporating judgments from people who assess generated outputs. Existing studies employ pairwise comparisons, rankings, ratings, or result selection to evaluate overall preference, semantic responsiveness, aesthetic quality, and other task-specific criteria [4, 24, 27, 62, 70, 83, 90, 118, 162]. These evaluations provide direct observations of human judgments without requiring a single automated metric to fully represent subjective preferences.

Nevertheless, human evaluation does not automatically constitute human-centered validation. Its relevance depends on who provides the judgments, which aspects of the outputs are assessed, and whether the evaluation setting reflects the intended use context [107]. For general image generation, participant preferences may provide useful evidence of perceived quality. However, personalized generation requires particular attention to the preferences of the intended users, while professional applications may additionally require expert judgments, structural validity, or domain-specific standards. Consequently, a favorable evaluation from a general participant population does not necessarily demonstrate that an individual user’s preferences or a professional task’s requirements have been satisfied.

Human-centered validation therefore requires alignment between the evaluation design and the human needs that optimization claims to address. Methods targeting personalized preferences should consider judgments from the relevant users, whereas methods targeting specialized tasks should incorporate appropriate contextual or expert assessments. More generally, the selected participants, evaluation criteria, and task settings should correspond to the intended optimization objectives [91, 98, 150, 170]. This distinction shifts evaluation from simply asking whether people prefer an output toward examining whether the system successfully supports the human goals it was designed to satisfy.

Human–Proxy Disagreement

Computational proxies and human judgments do not always produce consistent evaluations of generated results. An output may achieve a higher reward score, aesthetic rating, or text-image alignment score without receiving a corresponding improvement in human preference judgments. We characterize such observable inconsistencies between proxy-based evaluations and human-grounded assessments as Human–Proxy Disagreement. This disagreement may appear in differences in output rankings, preference selections, or assessments of generation quality.

Human–Proxy Disagreement is related to, but distinct from, the Human–Proxy Gap discussed in § 5. The Human–Proxy Gap describes a potential structural mismatch between human intent and its computational representation, whereas Human–Proxy Disagreement concerns differences that can be observed when computational evaluators and human participants assess generated outcomes. Such disagreements may arise because automated proxies emphasize selected attributes while users additionally consider structural validity, creativity, individual preferences, or task context.

Differences in training distributions, evaluation criteria, and target users may further affect the correspondence between learned evaluators and actual human judgments [132].

Importantly, disagreement does not necessarily imply that a computational proxy is entirely invalid, nor does every disagreement demonstrate an underlying failure of intent representation. Human judgments may themselves vary across participants and contexts. Nevertheless, systematic discrepancies between automated scores and relevant human evaluations provide useful evidence for identifying the limitations of computational proxies. When improvements in automated metrics are not accompanied by corresponding gains in human judgments, it becomes necessary to examine which aspects of human intent the optimization and evaluation objectives actually capture. Thus, Human–Proxy Disagreement offers an evaluation-level perspective on the broader challenge of validating human-intent alignment.

Beyond Static Evaluation: Interactive and Longitudinal Gaps

A recurring limitation in the reviewed literature is the emphasis on static comparisons of generated outputs. Human evaluations frequently involve one-time ratings or preference selections over completed images, while interactive processes through which users develop, revise, and coordinate their needs receive less explicit attention [72]. Such evaluations can establish preferences for particular outputs, but provide more limited evidence about how effectively a generative system supports users throughout an ongoing interaction.

This limitation is particularly relevant to personalized and exploratory generation. Users may begin with incomplete or uncertain intentions and progressively clarify their goals by observing, comparing, and modifying generated results. A related study of generative AI in personal health informatics reports a comparable interaction pattern in another domain: users formulate and refine queries, reflect on AI-generated recommendations, and sometimes abandon queries when responses do not meet their personal goals [17]. Outputs from earlier rounds can influence subsequent preferences, meaning that successful alignment may depend not only on the quality of a final image but also on whether the system accommodates evolving objectives. Evaluating these interactions therefore requires attention to how users revise their requests, exercise control, explore alternatives, and assess intermediate outcomes across multiple generation rounds.

Longitudinal evaluation introduces another important perspective. Users’ trust, perceived control, acceptance, and preferences may change as they gain experience with a generative system. Short-term evaluations cannot fully establish how these factors develop during repeated or prolonged use. Accordingly, future human-centered evaluation should complement static output assessments with interactive and longitudinal evidence, examining not only whether an individual result is preferred but also whether the system continues to support users’ goals as their needs and interaction contexts evolve.

Overall, validating human-intent alignment requires considering the complementary strengths and limitations of automated metrics, human judgments, and interaction-based assessments. Automated evaluations reveal measurable improvements, human-grounded assessments examine their relevance to actual preferences, and interactive or longitudinal evaluations can provide additional evidence of how well these improvements support users over time. Together, these perspectives offer a more comprehensive basis for determining whether diffusion model optimization genuinely advances human-centered outcomes.

Limitations and Future Work

This review has several limitations. First, our analysis synthesizes evidence reported in existing studies rather than providing new experiments or user studies. Consequently, our findings characterize reported optimization mechanisms, evaluation practices, and recurring tensions, but do not establish new causal relationships between optimization strategies and user experience. In particular, outcomes related to human agency, evolving intent, creative exploration, and long-term satisfaction are not consistently examined in the reviewed literature. Future empirical studies should investigate whether iterative alignment, adaptive objective coordination, and selective intervention can improve user outcomes in multi-turn generation, personalized creation, and sustained interaction.

Second, the scope of human-centered diffusion model optimization depends on the operational definition and inclusion criteria adopted in this review. Some studies optimize computational rewards related to human preferences without explicitly addressing human-centered interaction, while others provide meaningful user control without performing the preference-driven optimization required for inclusion in our core corpus. Although these boundaries help maintain a focused analytical scope, they may exclude adjacent work that offers relevant insights. Future reviews could examine how alternative inclusion criteria affect the characterization of this research area.

Third, the review is subject to limitations associated with literature retrieval and thematic synthesis. Differences in terminology across HCI, computer vision, and machine learning may affect which studies are identified through database searches and citation tracking. The interpretation and grouping of studies into analytical categories also involve methodological judgments, particularly when individual methods span multiple optimization mechanisms or forms of human involvement. These limitations should be considered when interpreting the resulting framework and the generality of the identified research themes.

Finally, the field is evolving rapidly. New generative architectures, preference-learning mechanisms, multimodal evaluators, and forms of human–AI interaction may introduce additional pathways through which human intent influences optimization. Accordingly, the categories and conceptual framework presented here should be understood as an analytical account of the reviewed literature rather than an exhaustive or closed taxonomy. Future work should revisit these relationships as new evidence emerges and investigate how the framework applies to increasingly interactive and personalized generative systems. Building on these limitations and emerging opportunities, Figure 8 summarizes four interconnected directions for future research: richer intent modeling, adaptive objective coordination, human-centered evaluation, and agency, safety, and deployment.

Figure 8. Future research agenda for human-centered diffusion model optimization. The agenda identifies four interconnected research directions: (1). Richer Intent Modeling, focusing on multimodal and context-aware intent representation and the modeling of evolving user preferences; (2). Adaptive Objective Coordination, addressing dynamic weighting of competing human-centered goals and selective intervention across model-level and generation-level optimization; (3). Human-Centered Evaluation, emphasizing human-grounded, domain-specific validation and interactive, longitudinal evaluation protocols; and (4). Agency, Safety, and Deployment, highlighting transparent controls, system inspectability, user override, real-world deployment, safety constraints, and feedback loops. Together, these directions outline a research agenda for developing more adaptive, interpretable, and human-aligned diffusion optimization systems.

Conclusion

This paper presents a systematic review of human-centered diffusion model optimization, examining how human intent is expressed, represented, translated into computational objectives, executed through optimization mechanisms, and evaluated against human needs. Rather than organizing existing research solely by algorithm type, we analyze four interconnected dimensions: intent representation, optimization mechanisms, failure modes and trade-offs, and human-centered validation.

Our synthesis highlights that computational objectives may capture only selected aspects of human intent, creating a potential Human–Proxy Gap that can be amplified during optimization. Model-level and generation-level optimization offer different mechanisms for executing these objectives and different opportunities for user intervention. Meanwhile, optimization may introduce trade-offs involving generative diversity, creative exploration, human agency, and system costs. Existing automated and human-grounded evaluations provide complementary evidence, but limitations remain in verifying whether computational improvements correspond to actual user satisfaction, particularly across interactive and longitudinal settings.

These findings motivate a human-centered perspective in which computational objectives remain open to reassessment, optimization interventions respond to task and user needs, and evaluation considers both generated results and interaction experiences. By connecting intent representation, optimization execution, and human-centered validation within a unified conceptual framework, this review provides a foundation for future research on generative systems that move beyond optimizing predefined proxies toward maintaining alignment with evolving human intent.

References

References

  1. [1]Emanuele Aiello, Umberto Michieli, Diego Valsesia, Mete Ozay, and Enrico Magli. 2025. DreamCache: Finetuning-Free Lightweight Personalized Image Generation via Feature Caching. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  2. [2]Saleema Amershi, Dan Weld, Michaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi Iqbal, Paul N Bennett, Kori Inkpen, et al. 2019. Guidelines for human-AI interaction. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–13.
  3. [3]Jingkun An, Yinghao Zhu, Zongjian Li, Enshen Zhou, Haoran Feng, Xijie Huang, Bohua Chen, Yemin Shi, and Chengwei Pan. 2025. AGFSync: Leveraging AI-Generated Feedback for Preference Optimization in Text-to-Image Generation. In Proceedings of the Aaai Conference on Artificial Intelligence.
  4. [4]Omri Avrahami, Or Patashnik, Ohad Fried, Egor Nemchinov, Kfir Aberman, Dani Lischinski, and Daniel Cohen-Or. 2025. Stable Flow: Vital Layers for Training-Free Image Editing. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).DOI
  5. [5]Hyojin Bangh, Caroline Chan, Frédo Durand, and Phillip Isola. 2025. Cycle Consistency as Reward: Learning Image-Text Alignment Without Human Preferences. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV).
  6. [6]Arpit Bansal, Hong-Min Chu, Avi Schwarzschild, Roni H. Segupta, Micah Goldblum, Jonas Geiping, and Tom Goldstein. 2024. Universal guidance for diffusion models. In International Conference on Learning Representations, Vol. 2024. 51304–51323.
  7. [7]Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. 2023. Training diffusion models with reinforcement learning. arXiv preprint arXiv:2305.13301 (2023).arxiv.org/abs/2305.13301
  8. [8]Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. 2024. Training diffusion models with reinforcement learning. In International Conference on Learning Representations, Vol. 2024. 4965–4987.
  9. [9]Stephen Brade, Bryan Wang, Mauricio Sousa, Regeer Oore, and Tovi Grossman. 2023. Promptify: Text-to-image generation through interactive prompt exploration with large language models. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. 1–14.arxiv.org/abs/2304.09337
  10. [10]Shengqi Cai, Eric Ryan Chan, Yunzhi Zhang, Leonidas Guibas, Jiajun Wu, and Gordon Wetzstein. 2025. Diffusion Self-Distillation for Zero-Shot Customized Image Generation. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  11. [11]Pu Cao, Feng Zhou, Qing Song, and Lu Yang. 2025. Controllable generation with text-to-image diffusion models: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence (2025).
  12. [12]Yining Cao, Yiyi Huang, Anh Truong, Hijung Valentina Shin, and Haijun Xia. 2025. Compositional structures as substrates for human-ai co-creation environment: A design approach and a case study. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–25.arxiv.org/abs/2503.04103
  13. [13]Liuqing Chen, Shuhong Xiao, Yunnong Chen, Yaxuan Song, Ruoyu Wu, and Lingyun Sun. 2024. ChatScratch: An AI-augmented system toward autonomous visual programming learning for children aged 6-12. In Proceedings of the 2024 CHI conference on human factors in computing systems. 1–19.DOI
  14. [14]Min Cheng, Fatemeh Doudi, Dileep Kalathil, Mohammad Ghavamzadeh, and Panganamala Kumar. 2026. Diffusion blend: Inference-time multi-preference alignment for diffusion models. In International Conference on Learning Representations, Vol. 2026. 51151–51181.arxiv.org/abs/2505.18547
  15. [15]Wonwoong Cho, Yan-Ying Chen, Matthew Klenk, David I. Inouye, and Yanxia Zhang. 2025. Att-Adapter: A Robust and Precise Domain-Specific Multi-Attributes Ti2i Diffusion Adapter Via Conditional Variational Autoencoder. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV).arxiv.org/abs/2503.11937
  16. [16]DaEun Choi, Sumin Hong, Jeongeon Park, John Joon Young Chung, and Juho Kim. 2024. Creativeconnect: Supporting reference recombination for graphic design ideation with generative ai. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–25.DOI
  17. [17]Shaan Chopra, Katherine Juarez, James Fogarty, and Sean A Munson. 2025. Engagements with generative AI and personal health informatics: opportunities for planning, tracking, reflecting, and acting around personal health data. Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies 9, 3 (2025), 1–33.DOI
  18. [18]Florinel-Alin Croitoru, Vlad Hondru, Radu Tudor Ionescu, Nicu Sebe, and Mubarak Shah. 2025. Curriculum Direct Preference Optimization for Diffusion and Consistency Models. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  19. [19]Meihua Dang, Anikait Singh, Linqi Zhou, Stefano Ermon, and Jiaming Song. 2025. Personalized Preference Fine-tuning of Diffusion Models. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).arxiv.org/abs/2501.06655
  20. [20]Hanhui Deng, Jianan Jiang, Zhiwang Yu, Jinhui Ouyang, and Di Wu. 2024. CrossGAI: A cross-device generative AI framework for collaborative fashion design. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 8, 1 (2024), 1–27.
  21. [21]Priya Dhawka, Lauren Perera, and Wesley Willett. 2024. Better little people pictures: Generative creation of demographically diverse anthropographics. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–14.
  22. [22]Guanqi Ding, Chengyu Yang, Shuhui Wang, Xincheng Li, Jinzhe Zhang, Xin Jin, and Qingming Huang. 2025. Dis2Booth: Learning Image Distribution with Disentangled Features for Text-to-Image Diffusion Models. In Proceedings of the Aaai Conference on Artificial Intelligence.
  23. [23]Xianzhe Fan, Zihan Wu, Chun Yu, Fenggui Rao, Weinan Shi, and Teng Tu. 2024. ContextCam: Bridging context awareness with creative human-AI image co-creation. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–17.DOI
  24. [24]Fu Feng, Yucheng Xie, Xu Yang, Jing Wang, and Xin Geng. 2025. Redefining <Creative> in Dictionary: Towards an Enhanced Semantic Understanding of Creative Generation. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  25. [25]Minghao Fu, Guo-Hua Wang, Liangfu Cao, Qing-Guo Chen, Zhao Xu, Weihua Luo, and Kaifu Zhang. 2025. CHATS: Combining human-aligned optimization and test-time sampling for text-to-image generation. arXiv preprint arXiv:2502.12579 (2025).arxiv.org/abs/2502.12579
  26. [26]Yaozhong Gan, Renye Yan, Xiaoyang Tan, Zhe Wu, and Junliang Xing. 2024. Transductive off-policy proximal policy optimization. arXiv preprint arXiv:2406.03894 (2024).arxiv.org/abs/2406.03894
  27. [27]Rohit Gandikota, Joanna Materzyńska, Jaden Fiotto-Kaufman, and David Bau. 2023. Erasing Concepts from Diffusion Models. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV).arxiv.org/abs/2303.07345
  28. [28]Rohit Gandikota, Zongze Wu, Richard Zhang, David Bau, Eli Shechtman, and Nick Kolkin. 2025. SliderSpace: Decomposing the Visual Capabilities of Diffusion Models. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV).arxiv.org/abs/2502.01639
  29. [29]Kazjon Grace, Francisco J Ibarolla, Jody Watts, Shu Takahashi, Parth Bhargava, and Eduardo Velloso. 2026. Do Entropic Measurements of the Diversity of AI-generated Images Match Human Judgement?. In Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems. 1–15.DOI
  30. [30]Jiayi Guo, Chuanhao Yan, Xingqian Xu, Yulin Wang, Kai Wang, Gao Huang, and Humphrey Shi. 2025. IMG: Calibrating Diffusion Models via Implicit Multimodal Guidance. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV).arxiv.org/abs/2509.26231
  31. [31]Qin Guo and Tianwei Lin. 2024. Focus on Your Instruction: Fine-grained and Multi-instruction Image Editing by Attention Modulation. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  32. [32]Xiefan Guo, Miaomiao Cui, Liefeng Bo, and Di Huang. 2025. ShortFT: Diffusion Model Alignment via Shortcut-Based Fine-Tuning. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV).arxiv.org/abs/2507.22604
  33. [33]Yuanhe Guo, Linxi Xie, Zhuoran Chen, Kangrui Yu, Ryan Po, Guandao Yang, Gordon Wetzstein, and Hongyi Wen. 2025. ImageGem: In-the-wild Generative Image Interaction Dataset for Generative Model Personalization. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV).arxiv.org/abs/2510.18433
  34. [34]Amir Hertz, Andrey Voynov, Shlomi Fruchter, and Daniel Cohen-Or. 2024. Style Aligned Image Generation via Shared Attention. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  35. [35]Jiwoo Hong, Sayak Paul, Noah Lee, Kashif Rasul, James Thorne, and Jongheon Jeong. 2026. Margin-Aware Preference Optimization for Aligning Diffusion Models Without Reference. In Proceedings of the Aaai Conference on Artificial Intelligence.
  36. [36]Yunqi Hong, Kuei-Chun Kao, Hengguang Zhou, and Cho-Jui Hsieh. 2026. Understanding reward hacking in text-to-image reinforcement learning. arXiv preprint arXiv:2601.03468 (2026).
  37. [37]Zijing Hu, Fengda Zhang, Long Chen, Kun Kuang, Jiahui Li, Kaifeng Gao, Jun Xiao, Xin Wang, and Wenwu Zhu. 2025. Towards Better Alignment: Training Diffusion Models with Reinforcement Learning Against Sparse Rewards. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).arxiv.org/abs/2503.11240
  38. [38]Junjia Huang, Pengxiang Yan, Jiyang Liu, Jie Wu, Zhao Wang, Yitong Wang, Liang Lin, and Guanbin Li. 2025. DreamFuse: Adaptive Image Fusion with Diffusion Transformer. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV).arxiv.org/abs/2504.08291
  39. [39]Junjia Huang, Pengxiang Yan, Jiyang Liu, Jie Wu, Zhao Wang, Yitong Wang, Xinglong Wu, Liang Lin, and Guanbin Li. 2026. DreamFuse: Toward Realistic and Seamless Image Fusion Across Diverse Scenarios. IEEE Transactions on Pattern Analysis and Machine Intelligence (2026).DOI
  40. [40]Mao Xun Huang, Brian J Chan, and Hen-Hsen Huang. 2025. SmartSpatial: Enhancing 3D Spatial Awareness in Stable Diffusion with a Novel Evaluation Framework. In Ijcai International Joint Conference on Artificial Intelligence.
  41. [41]Qihan Huang, Long Chan, Jinlong Liu, Wanggui He, Hao Jiang, Mingli Song, and Jie Song. 2025. PatchDPO: Patch-level DPO for Finetuning-free Personalized Image Generation. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  42. [42]Qihan Huang, Siming Fu, Jinlong Liu, Hao Jiang, Yipeng Yu, and Jie Song. 2025. Resolving Multi-Condition Confusion for Finetuning-Free Personalized Image Generation. In Proceedings of the Aaai Conference on Artificial Intelligence.
  43. [43]Rong Huang, Haichuan Lin, Chuanzhang Chen, Kang Zhang, and Wei Zeng. 2024. PlantoGraphy: Incorporating iterative design process into generative artificial intelligence for landscape rendering. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–19.DOI
  44. [44]Mina Huh, Ding Li, Kim Pimmel, Hijung Valentina Shin, Amy Pavel, and Mira Dontcheva. 2025. Videodiff: Human-ai video co-creation with alternatives. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–19.DOI
  45. [45]Sadeep Jayasumana, Srikumar Ramalingam, Andreas Veit, Daniel Glasner, Ayan Chakrabarti, and Sanjiv Kumar. 2024. Rethinking fid: Towards a better evaluation metric for image generation. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 9307–9315.arxiv.org/abs/2401.09603
  46. [46]Rohit Jena, Ali Taghibakhshi, Sahil Jain, Gerald Shen, Nima Tajbakhsh, and Arash Vahdat. 2024. Elucidating optimal reward-diversity tradeoffs in text-to-image diffusion models. arXiv preprint arXiv:2409.06493 (2024).arxiv.org/abs/2409.06493
  47. [47]Jaeseok Jeong, Junho Kim, Gayoung Lee, Yunje Cho, and Youngung Uh. 2025. Stylekeeper: Prevent Content Leakage using Negative Visual Query Guidance. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV).arxiv.org/abs/2510.06827
  48. [48]Zhiwei Jia, Yuesong Nan, Huixi Zhao, and Gengdai Liu. 2025. Reward Fine-Tuning Two-Step Diffusion Models via Learning Differentiable Latent-Space Surrogate Reward. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  49. [49]Jianan Jiang, Di Wu, Hanhui Deng, Yidan Long, Wenyi Tang, Xiang Li, Can Liu, Zhanpeng Jin, Wenlei Zhang, and Tangquan Qi. 2024. Haigen: towards human-ai collaboration for facilitating creativity and style generation in fashion design. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 8, 3 (2024), 1–27.DOI
  50. [50]Jiaxiu Jiang, Yabo Zhang, Kailai Feng, Xiaohe Wu, Wenbo Li, Renjing Pei, Fan Li, and Wangmeng Zuo. 2025. MC2: Multi-concept Guidance for Customized Multi-concept Generation. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  51. [51]Rui Jiang, Guang-Cong Zheng, Teng Li, Tian-Rui Yang, Jing-Dong Wang, and Xi Li. 2024. A survey of multimodal controllable diffusion models. Journal of Computer Science and Technology 39, 3 (2024), 509–541.
  52. [52]Negar Kamali, Karyn Nakamura, Aakriti Kumar, Angelos Chatzimparmpas, Jessica Hullman, and Matthew Groh. 2025. Characterizing photorealism and artifacts in diffusion model-generated images. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–26.arxiv.org/abs/2502.11989
  53. [53]Junyong Kang, Seohyun Lim, Kyungjune Baek, and Hyunjung Shim. 2026. Rethinking Direct Preference Optimization in Diffusion Models. In Proceedings of the Aaai Conference on Artificial Intelligence.
  54. [54]Shyamgopal Karthik, Huseyin Coskun, Zeynep Akata, Sergey Tulyakov, Jian Ren, and Anil Kag. 2025. Scalable Ranked Preference Optimization for Text-To-Image Generation. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV).
  55. [55]Jeeyung Kim, Erfan Esmaeili, and Qiang Qiu. 2025. Text Embedding is Not All You Need: Attention Control for Text-to-Image Semantic Alignment with Text Self-Attention Maps. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  56. [56]Jaemin Kim, Bryan Sangwoo Kim, and Jong Chul Ye. 2025. Free2 Guide: Training-Free Text-to-Video Alignment Using Image LVLM. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV).
  57. [57]Yunji Kim, Jiyoung Lee, Jin-Hwa Kim, Jung-Woo Ha, and Jun-Yan Zhu. 2023. Dense Text-to-Image Generation with Attention Modulation. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV).
  58. [58]Jaegun Lee and Janghoon Choi. 2026. Flow-Multi: A Flow-Matching Multi-Reward Framework for Text-to-Image Generation. Sensors 26, 4 (2026), 1120.
  59. [59]Kyungmin Lee, Xiaohang Li, Qifei Wang, Junfeng He, Junjie Ke, Ming-Hsuan Yang, Irfan Essa, Jinwoo Shin, Feng Yang, and Yinxiao Li. 2025. Calibrated Multi-Preference Optimization for Aligning Diffusion Models. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  60. [60]Seung Won Lee, Tae Hee Jo, Semin Jin, Jiin Choi, Kyungwon Yun, Sergio Bromberg, Seonghoon Ban, and Kyung Hoon Hyun. 2024. The impact of sketch-guided vs. prompt-guided 3D generative AIs on the design exploration process. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–18.DOI
  61. [61]Baiqi Li, Zhiqi Lin, Deepak Pathak, Jiaxiao Li, Yixin Fei, Kewen Wu, Xide Xia, Pengchuan Zhang, Graham Neubig, and Deva Ramanan. 2024. Evaluating and improving compositional text-to-visual generation. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). IEEE, 5290–5301.arxiv.org/abs/2406.13743
  62. [62]Jialuo Liu, Wenhao Chai, Xingyu Fu, Haiyang Xu, and Saining Xie. 2025. Science-T2I: Addressing Scientific Illusions in Image Synthesis. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  63. [63]Xiaoming Li, Xinyu Hou, and Chen Chang. 2024. When StyleGAN Meets Stable Diffusion: a W+ Adapter for Personalized Image Generation. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  64. [64]Xiaomin Li, Yixuan Liu, Takashi Isobe, Xu Jia, Qinpeng Cui, Dong Zhou, Dong Liu, You He, Huchuan Lu, Zhongdao Wang, and Emad Barsoum. 2025. ReNeg: Learning Negative Embedding with Reward Guidance. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  65. [65]Yang Li, Songlin Yang, Xiaoxuan Han, Wei Wang, Jing Dong, Yueming Lyu, and Ziyu Xue. 2026. Instant Preference Alignment for Text-to-Image Diffusion Models. IEEE Transactions on Visualization and Computer Graphics (2026).
  66. [66]Yang Li, Songlin Yang, Wei Wang, and Jing Dong. 2025. Beyond Inserting: Learning Subject Embedding for Semantic-Fidelity Personalized Diffusion Generation. IEEE Transactions on Circuits and Systems for Video Technology (2025).
  67. [67]Zhipeng Li, Yi-Chi Liao, and Christian Holz. 2026. Preference-Guided Prompt Optimization for Text-to-Image Generation. In Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems. 1–29.
  68. [68]Youwen Liang, Junfeng He, Gang Li, Peizhao Li, Arseniy Klymovskiy, Nicholas Carolan, Jiao Sun, Jordi Pont-Tuset, Sarah Young, Feng Yang, Junjie Ke, Krishnamurthy Dj Dvijotham, Katherine M. Collins, Yiwen Luo, Yang Li, Kai J Kohlhoff, Deepak Ramachandran, and Vidhya Navalpakam. 2024. Rich Human Feedback for Text-to-Image Generation. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  69. [69]Zhanhao Liang, Yuhui Yuan, Shuyang Guo, Bohan Chen, Tiankai Hang, Mingxı Cheng, Ji Li, and Liang Zheng. 2025. Aesthetic Post-Training Diffusion Models from Generic Preferences with Step-by-step Preference Optimization. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  70. [70]Yuanze Lin, Yi-Wen Chen, Yi-Hsuan Tsai, Lu Jiang, and Ming-Hsuan Yang. 2024. Text-Driven Image Editing via Learnable Regions. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  71. [71]Buhua Liu, Shitong Shao, Bao Li, Lichen Bai, Zhiqiang Xu, Haoyi Xiong, James T Kwok, Sumi Helal, and Zeke Xie. 2026. Alignment of diffusion models: Fundamental challenges, and future. Comput. Surveys 58, 9 (2026), 1–37.
  72. [72]Chenxi Liu, Selena Ling, and Alec Jacobson. 2026. GimmBO: Interactive Generative Image Model Merging via Bayesian Optimization. ACM Transactions on Graphics (2026).
  73. [73]Fangyu Liu, Junliang Ye, Yikai Wang, Hanyang Wang, Zhengyi Wang, Jun Zhu, and Yueqi Duan. 2025. DreamReward-X: Boosting High-Quality 3D Generation with Human Preference Alignment. IEEE Transactions on Pattern Analysis and Machine Intelligence (2025).
  74. [74]Runtao Liu, Haoyou Wu, Ziqiang Zheng, Chen Wei, Yingqing He, Renjie Pi, and Qifeng Chen. 2025. VideoDPO: Omni-Preference Alignment for Video Diffusion Generation. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  75. [75]Yunhong Lu, Qichao Wang, Hengyuan Cao, Xierui Wang, Xiaoyin Xu, and Min Zhang. 2025. InPO: Inversion Preference Optimization with Reparameterized DDIM for Efficient Diffusion Model Alignment. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  76. [76]Yihong Luo, Tianyang Hu, Yifan Song, Jiacheng Sun, Zhenguo Li, and Jing Tang. 2025. Adding Additional Control to One-Step Diffusion with Joint Distribution Matching. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV).arxiv.org/abs/2503.06652
  77. [77]Zhengyao Lv, Tianlin Pan, Chenyang Si, Zhaoxi Chen, Wangmeng Zuo, Ziwei Liu, and Kwan-Yee K. Wong. 2025. Rethinking Cross-Modal Interaction in Multimodal Diffusion Transformer. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV).arxiv.org/abs/2506.07986
  78. [78]Mengyao Lyu, Yuhong Yang, Haiwen Hong, Hui Chen, Xuan Jin, Yuan He, Hui Xue, Jungong Han, and Guiguang Ding. 2024. One-dimensional Adapter to Rule Them All: Concepts, Diffusion Models and Erasing Applications. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  79. [79]Yuhang Ma, Xiaoshi Wu, Keqiang Sun, and Hongsheng Li. 2025. Hpsv3: Towards wide-spectrum human preference score. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 15086–15095.arxiv.org/abs/2508.03789
  80. [80]Atefeh Mahdavi Goloujeh, Anne Sullivan, and Brian Magerko. 2024. Is it AI or is it me? Understanding users’ prompt journey with text-to-image generative AI tools. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–13.
  81. [81]Sebastian Maier, Manuel Schneider, and Stefan Feuerriegel. 2026. Partnering with Generative AI: Experimental Evaluation of Model-Led and Human-Led Interaction in Human-AI Co-Creation. In Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems. 1–28.
  82. [82]Zichen Miao, Jiang Wang, Ze Wang, Zhengyuan Yang, Lijuan Wang, Qiang Qiu, and Zicheng Liu. 2024. Training Diffusion Models Towards Diverse Image Generation with Reinforcement Learning. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  83. [83]Haoran Mo, Yulin Shen, Edgar Simo-Serra, and Zeyu Wang. 2026. DoodleAssist: Progressive Interactive Line Art Generation With Latent Distribution Alignment. IEEE Transactions on Visualization and Computer Graphics (2026).DOI
  84. [84]Wenyi Mo, Tianyu Zhang, Yalong Bai, Bing Su, Ji-Rong Wen, and Qing Yang. 2024. Dynamic Prompt Optimizing for Text-to-Image Generation. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).arxiv.org/abs/2404.04095
  85. [85]Steven Moore, Q Vera Liao, and Hariharan Subramonyam. 2023. FallureNotes: Supporting designers in understanding the limits of AI models for computer vision tasks. In Proceedings of the 2023 CHI conference on human factors in computing systems. 1–19.arxiv.org/abs/2302.11703
  86. [86]Sunung Mun, Jinhwan Nam, Sunghyun Cho, and Jungseul Ok. 2025. Addressing Text Embedding Leakage in Diffusion-Based Image Editing. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV).
  87. [87]Sanghyeon Na, Yonggyu Kim, and Hyunjoon Lee. 2025. Boost Your Human Image Generation Model via Direct Preference Optimization. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  88. [88]Syeda Masooma Naqvi, Ruichen He, and Harmanpreet Kaur. 2025. Catalyst for creativity or a hollow trend?: A cross-level perspective on the role of generative AI in design. In Proceedings of the 2025 CHI conference on human factors in computing systems. 1–16.
  89. [89]Mayu Otani, Riku Togashi, Yu Sawai, Ryosuke Ishigami, Yuta Nakashima, Esa Rahtu, Janne Heikkilä, and Shin’ichi Satoh. 2023. Toward verifiable and reproducible human evaluation for text-to-image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 14277–14286.arxiv.org/abs/2304.01816
  90. [90]Karran Pandey, Paul Guerrero, Mattheus Gadelha, Yannick Hold-Geoffroy, Karan Singh, and Niloy J. Mitra. 2024. Diffusion Handles Enabling 3D Edits for Diffusion Models by Lifting Activations to 3D. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  91. [91]Rishubh Parihar, Vaibhav Agrawal, Sachidanand VS, and R. Venkatesh Babu. 2025. Compass Control: Multi Object Orientation Control for Text-to-Image Generation. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).arxiv.org/abs/2504.06752
  92. [92]Sunghyun Park, Seokeon Choi, Hyoungwoo Park, and Sungrock Yun. 2025. Steering Guidance for Personalized Text-to-Image Diffusion Models. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV).arxiv.org/abs/2508.00319
  93. [93]Xu Peng, Junwei Zhu, Boyuan Jiang, Ying Tai, Donghao Luo, Jiangning Zhang, Wei Lin, Taisong Jin, Chengjie Wang, and Rongrong Ji. 2024. PortraitBooth: A Versatile Portrait Model for Fast Identity-Preserved Personalization. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  94. [94]Yuqi Peng, Lingtao Zheng, Yufeng Yang, Yi Huang, Mingfu Yan, Jianzhuang Liu, and Shifeng Chen. 2026. TARA: Token-Aware LoRA for Composable Personalization in Diffusion Models. In Proceedings of the Aaai Conference on Artificial Intelligence.
  95. [95]Sagi Polaczek, Yuval Alaluf, Elad Richardson, Yalel Venk, and Daniel Cohen-Or. 2025. NeuralSVG: An Implicit Representation for Text-to-Vector Generation. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV).arxiv.org/abs/2501.03992
  96. [96]Julien Pouget, Sitong Wang, and Lydia B Chilton. 2025. Copying style, extracting value: Illustrators’ perception of ai style transfer and its impact on creative labor. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–16.
  97. [97]Mihir Prabhudesai, Anirudh Goyal, Deepak Pathak, and Katerina Fragkiadaki. 2023. Aligning Text-to-Image Diffusion Models with Reward Backpropagation. arXiv preprint arXiv:2310.03739 (2023).arxiv.org/abs/2310.03739
  98. [98]Yuming Qiao, Fanyi Wang, Jingwen Su, Yanhao Zhang, Yunjie Yu, Siyu Wu, and Guo-Jun Qi. 2024. BARET: Balanced Attention Based Real Image Editing Driven by Target-Text Inversion. In Proceedings of the Aaai Conference on Artificial Intelligence.
  99. [99]Nina Rajcic, Maria Teresa Llano Rodriguez, and Jon McCormack. 2024. Towards a diffractive analysis of prompt-based generative AI. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–15.DOI
  100. [100]Nathaniel Ruiz, Yuanzhen Li, Varun Jampani, Wei Wei, Tingbo Hou, Yael Pritch, Neal Wadhwa, Michael Rubinstein, and Kfir Aberman. 2024. HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Models. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  101. [101]Nathaniel Ruiz, Yuanzhen Li, Neal Wadhwa, Yael Pritch, Michael Rubinstein, David E. Jacobs, and Shlomi Fruchter. 2025. Magic Insert: Style-Aware Drag-And-Drop. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV).
  102. [102]Hanshu Shen, Lyukesheng Shen, Wenqi Wu, and Kejun Zhang. 2025. Ideationweb: Tracking the evolution of design ideas in human-ai co-creation. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–19.DOI
  103. [103]Leixian Shen, Yifang Wang, Huamin Qiu, Xing Xie, and Haotian Li. 2026. Interaction-augmented instruction: Modeling the synergy of prompts and interactions in human-general collaboration. In Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems. 1–21.DOI
  104. [104]Donald Shenaj, Ondrej Bohdal, Mete Ozay, Pietro Zanuttigh, and Umberto Michieli. 2025. LoRA.rar: Learning to Merge LoRAs via Hypernetworks for Subject-Style Conditioned Image Generation. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV).
  105. [105]Christian Simon, Masato Ishii, Akio Hayakawa, Zhi Zhong, Shusuke Takahashi, Takashi Shibuya, and Yuki Mitsufuji. 2025. TITAN-Guide: Taming Inference-Time Alignment for Guided Text-to-Video Diffusion Models. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV).arxiv.org/abs/2508.00289
  106. [106]Hannu Simonen, Atte Kiviniemi, Hannah Johnston, Helena Barranha, and Jonas Oppenlaender. 2026. An exploration of default images in text-to-image generation. In Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems. 1–22.DOI
  107. [107]Jaskirat Singh, Stephen Gould, and Liang Zheng. 2023. High-Fidelity Guided Image Synthesis with Latent Diffusion Models. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  108. [108]James Smith, Shm Garanqao Almeida, Timothy J Ayeni, Anya Agarwal, and Bjoern Hartmann. 2026. Noise Pilot: Enabling Artistic Workflow Composition with Diffusion-Based Image Generation. In Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems. 1–20.DOI
  109. [109]Vera Soboleva, Aibek Alanov, Andrey Kuznetsov, and Konstantin Sobolev. 2026. T-LoRA: Single Image Diffusion Model Customization Without Overfitting. In Proceedings of the Aaai Conference on Artificial Intelligence.
  110. [110]Kihoon Son, DaEun Choi, Tae Soo Kim, Young-Ho Kim, and Juho Kim. 2024. Genquery: Supporting expressive visual search with generative models. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–19.
  111. [111]Hari Subramonyam, Roy Pea, Christopher Pondoc, Maneesh Agrawala, and Colleen Seifert. 2024. Bridging the gulf of envisioning: Cognitive challenges in prompt based interactions with LLMs. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–19.DOI
  112. [112]Haoyuan Sun, Bo Xia, Yongzhe Chang, and Xueqian Wang. 2025. Generalizing Alignment Paradigm of Text-to-Image Generation with Preferences Through f-Divergence Minimization. In Proceedings of the Aaai Conference on Artificial Intelligence.
  113. [113]Yuzheng Tan, Yuan He, Yao Zhu, Tianlin Huo, Huqian Yan, Hang Su, Shuxin Zhang, and Guangueng Hu. 2026. Dual-Seed Evolutionary Algorithm for Noise Optimization in Diffusion Models. In Proceedings of the Aaai Conference on Artificial Intelligence.
  114. [114]Khoi Trinh, Scott Seidenberger, Raveen Wijewickrama, Murtuza Jadliwala, and Anindya Maiti. 2025. A picture is worth a thousand prompts? efficacy of iterative human-driven prompt refinement in image regeneration tasks. arXiv preprint arXiv:2504.20340 (2025).arxiv.org/abs/2504.20340
  115. [115]Soobin Um and Jong Chul Ye. 2025. Minority-Focused Text-to-Image Generation via Prompt Optimization. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  116. [116]Samangi Wadinambiarchchi, Ryan M Kelly, Saumya Pareek, Qiuszhi Zhou, and Eduardo Velloso. 2024. The effects of generative AI on design fixation and divergent thinking. In Proceedings of the 2024 CHI conference on human factors in computing systems. 1–18.DOI
  117. [117]Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. 2024. Diffusion model alignment using direct preference optimization. In Computer Vision and Pattern Recognition. 8228–8238.
  118. [118]Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. 2024. Diffusion Model Alignment Using Direct Preference Optimization. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  119. [119]Bram Wallace, Akash Gokul, Stefano Ermon, and Nikhil Naik. 2023. End-to-End Diffusion Latent Optimization Improves Classifier Guidance. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV).arxiv.org/abs/2303.13703
  120. [120]Fu-Yun Wang, Keqiang Sun, Yao Teng, Xihui Liu, Jiale Yuan, Jiaming Song, and Hongsheng Li. 2026. Self-NPO: Data-Free Diffusion Model Enhancement via Truncated Diffusion Fine-Tuning. In Proceedings of the Aaai Conference on Artificial Intelligence.
  121. [121]Shulei Wang, Wang Lin, Hai Huang, Hanting Wang, Sihang Cai, WenKang Han, Tao Jin, Jingyuan Chen, Jiacheng Sun, Jieming Zhu, and Zhou Zhao. 2025. Towards Transformer-Based Aligned Generation with Self-Coherence Guidance. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).arxiv.org/abs/2503.17675
  122. [122]Shengzhi Wang, Yingkang Zhong, Jiangchuan Mu, Kai Wu, Mingliang Xiong, Wen Fang, Mingqing Liu, Hao Deng, Bin He, Gang Li, and Qingwen Liu. 2025. Align-A-Video: Deterministic Reward Tuning of Image Diffusion Models for Consistent Video Editing. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).DOI
  123. [123]Shun-Yu Wang, Wei-Chung Su, Serena Chen, Ching-Yi Tsai, Marta Misztal, Katherine M Cheng, Alwena Lin, Yu Chen, and Mike Y Chen. 2024. Roomdreaming: Generative-AI approach to facilitating iterative, preliminary interior design exploration. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–20.DOI
  124. [124]Weitao Wang, Haoran Xu, Yuxiao Yang, Zhifang Liu, Jun Meng, and Haoqian Wang. 2025. MVReward: Better Aligning and Evaluating Multi-View Diffusion Models with Human Preferences. In Proceedings of the Aaai Conference on Artificial Intelligence.
  125. [125]Yunlong Wang, Shuyuan Shen, and Brian Y Lim. 2023. Reprompt: Automatic prompt editing to refine ai-generative art towards precise expressions. In Proceedings of the 2023 CHI conference on human factors in computing systems. 1–29.DOI
  126. [126]Zhijie Wang, Yuheng Huang, Da Song, Lei Ma, and Tianyi Zhang. 2024. Promptcharm: Text-to-image generation through multi-modal prompting and refinement. In Proceedings of the 2024 CHI conference on human factors in computing systems. 1–21.DOI
  127. [127]Tianyi Wei, Dongdong Chen, Yifan Zhou, and Xingang Pan. 2026. Enhancing MMDiT-Based Text-to-Image Models for Similar Subject Generation. IEEE Transactions on Pattern Analysis and Machine Intelligence (2026).
  128. [128]Olivia Wiles, Chuhan Zhang, Isabela Albuquerque, Ivana Kajić, Su Wang, Emanuele Bugliariello, Yasumasa Onoe, Pinelopi Papalampidi, Ira Ktena, Christopher Knutsen, et al. 2025. Revisiting text-to-image evaluation with gecko: on metrics, prompts, and human rating. In International Conference on Learning Representations, Vol. 2025. 272–287.arxiv.org/abs/2404.16820
  129. [129]Fuxiang Wu, Liu Liu, Fusheng Hao, Chengqun Song, Dacheng Tao, and Jun Cheng. 2026. NoisePO: Efficient Semantic Noise Generation and Ranking for Diffusion-Based Text-to-Image Synthesis. IEEE Transactions on Image Processing (2026).DOI
  130. [130]Sihao Wu, Xiaonan Si, Chi Xing, Jianhong Wang, Gaojie Jin, Guangliang Cheng, and Xiaowei Huang. 2026. Preference alignment on diffusion models: A comprehensive survey for image generation and editing. Computer Science Review 61 (2026), 100900.
  131. [131]Xun Wu, Shaohan Huang, Lingjie Jiang, and Furu Wei. 2025. Rethinking DPO-Style Diffusion Aligning Frameworks. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV).DOI
  132. [132]Xiaoshi Wu, Keqiang Sun, Feng Zhu, Rui Zhao, and Hongsheng Li. 2023. Human Preference Score: Better Aligning Text-to-image Models with Human Preference. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV).arxiv.org/abs/2303.14420
  133. [133]Guangxuan Xiao, Tianwei Yin, William T. Freeman, Frédo Durand, and Song Han. 2025. FastComposer: Tuning-Free Multi-subject Image Generation with Localized Attention. International Journal of Computer Vision (2025).arxiv.org/abs/2305.10431
  134. [134]Shishi Xiao, Liangwei Wang, Xiaoj una Ma, and Wei Zeng. 2024. TypeDance: Creating semantic typographic logos from image through personalized generation. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–18.DOI
  135. [135]Xin Xie and Dong Gong. 2025. DyMO: Training-Free Diffusion Model Alignment with Dynamic Multi-Objective Scheduling. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  136. [136]Jiang Xin, Xiaonan Fang, Yueling Zhu, Ju Ren, and Yaoxue Zhang. 2025. C²D²: Context-Aware Concept Decomposition for Personalized Text-to-Image Synthesis. IEEE Transactions on Visualization and Computer Graphics (2025).DOI
  137. [137]Xiaoying Xing, Avinab Saha, Junfeng He, Susan Hao, Paul Vicol, Moonkyung Ryu, Gang Li, Sahil Singla, Sarah Young, Yinxiao Li, Feng Yang, and Deepak Ramachandran. 2025. Focus-NF: Region-Aware Fine-Tuning for Text-to-Image Generation. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).DOI
  138. [138]Ximing Xing, Qian Yu, Chuang Wang, Haitao Zhou, Jing Zhang, and Dong Xu. 2025. SVGDreamer++: Advancing Editability and Diversity in Text-Guided SVG Generation. IEEE Transactions on Pattern Analysis and Machine Intelligence (2025).
  139. [139]Ximing Xing, Haitao Zhou, Chuang Wang, Jing Zhang, Dong Xu, and Qian Yu. 2024. SVGDreamer: Text Guided SVG Generation with Diffusion Model. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  140. [140]Jiangzheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. 2023. Imagereward: Learning and evaluating human preferences for text-to-image generation. Advances in Neural Information Processing Systems (2023).arxiv.org/abs/2304.05977
  141. [141]Yifei Xu, Xiaolong Xu, Honghao Gao, and Fu Xiao. 2024. SGDM: An Adaptive Style-Guided Diffusion Model for Personalized Text to Image Generation. IEEE Transactions on Multimedia (2024).DOI
  142. [142]Renye Yan, Jikang Cheng, Yaozhong Gan, Shikun Sun, You Wu, Yunfan Yang, Ling Liang, Jinlong Lin, Yeshuang Zhu, Jie Zhou, Jinchao Zhang, Junliang Xing, Yimao Cai, and Ru Huang. 2025. Entropy-Adaptive Diffusion Policy Optimization with Dynamic Step Alignment. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV).DOI
  143. [143]Renye Yan, Jikang Cheng, Shikun Sun, Yi Sun, Wei Peng, Yaozhong Gan, You Wu, Ling Liang, Junliang Xing, and Yimao Cai. [n. d.] MultiTune: Phase-Aware Multi-Objective Optimization for Diffusion Models. ([n. d.]).
  144. [144]Renye Yan, Jikang Cheng, Shikun Sun, Zhiyuan Yan, Yaozhong Gan, Ling Liang, Jinchao Zhang, and Yimao Cai. 2026. LitExplorer: Training-Free Diffusion Guidance with Adaptive Exploration-Filtering Framework. (2026).
  145. [145]Renye Yan, Jikang Cheng, You Wu, Bojin Huang, Wei Peng, Zongwei Wang, Ling Liang, and Yimao Cai. 2026. SwiftExplorer: Training-free Diffusion Model Alignment with Swift Diversity Exploration. arXiv preprint arXiv:2609.06651 (2026).arxiv.org/abs/2609.06651
  146. [146]Renye Yan, Jikang Cheng, You Wu, Wei Peng, Zongwei Wang, Ling Liang, and Yimao Cai. 2026. Explore or Converge? Stage-Guided Per-Step Optimization for Diffusion Models. arXiv preprint arXiv:2608.06768 (2026).arxiv.org/abs/2608.06768
  147. [147]Renye Yan, Jikang Cheng, You Wu, Wei Peng, Zongwei Wang, Ling Liang, and Yimao Cai. 2026. PAST: Prompt-Adaptive Sampling Termination for Efficient Diffusion Model. arXiv preprint arXiv:2608.06794 (2026).arxiv.org/abs/2608.06794
  148. [148]Renye Yan, Junyi Zhan, Jikang Cheng, and Wei Peng. 2026. Timing Matters Twice: When to Query and When to Skip in Flow-Matching On-Policy Distillation. arXiv preprint arXiv:2610.34191 (2026).arxiv.org/abs/2610.34191
  149. [149]Renye Yan, Jialun Zhong, Jikang Cheng, Yi Gao, Yanzeng Li, and Ling Liang. 2026. Beyond Explicit Intent: Discovering and Shaping Artistic Intent through Human-AI Co-Creation. arXiv:2610.24010 [cs.AI] https://arxiv.org/abs/2610.24010
  150. [150]Han Yang, Chuangguang Yang, Qiuli Wang, Zhulin An, Weiliun Feng, Libo Huang, and Yongjun Xu. 2025. Multi-party Collaborative Attention Control for Image Customization. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).arxiv.org/abs/2505.01428
  151. [151]Jingyuan Yang, Jiawei Feng, and Hui Huang. 2024. EmoGen: Emotional Image Content Generation with Text-to-Image Diffusion Models. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).arxiv.org/abs/2401.04608
  152. [152]Kai Yang, Jian Tao, Jiafei Lyu, Chunj iang Ge, Jiaxin Chen, Weih an Shen, Xiaolong Zhu, and Xiu Li. 2024. Using Human Feedback to Fine-tune Diffusion Models without any Reward Model. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  153. [153]Shuya Yang, Shaozhe Hao, Yukang Cao, and Kwan-Yee K. Wong. 2025. ArtiFade: Learning to Generate High-quality Subject from Blemished Images. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  154. [154]Xiang Yang, Xiaohui Li, Yuke Wang, and Ninghao Liu. 2026. Optimizing Trust and Safety Regions for Text-to-Image Generation in High-Dimensional Manifold Spaces. IEEE Transactions on Pattern Analysis and Machine Intelligence (2026).DOI
  155. [155]Ziliyu Ye, Zhiyang Chen, Tiancheng Li, Zemin Huang, Weijian Luo, and Guo-Jun Qi. 2025. Schedule On the Fly: Diffusion Time Prediction for Faster and Better Image Generation. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  156. [156]Hangjie Yuan, Shiwei Zhang, Xiang Wang, Yujie Wei, Tao Feng, Yining Pan, Yingya Zhang, Ziwei Liu, Samuel Albanie, and Dong Ni. 2024. InstructVideo: Instructing Video Diffusion Models with Human Feedback. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  157. [157]Taeyoung Yun, Dinghuai Zhang, Jinkyoo Park, and Ling Pan. 2025. Learning to Sample Effective and Diverse Prompts for Text-to-Image Generation. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  158. [158]Xingchen Zeng, Ziyao Gao, Yilin Ye, and Wei Zeng. 2024. IntentTuner: an interactive framework for integrating human intentions in fine-tuning text-to-image generative models. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–18.
  159. [159]Chenshuang Zhang, Chaoning Zhang, Mengchun Zhang, In So Kweon, and Junmo Kim. 2023. Text-to-image diffusion models in generative ai: A survey. arXiv preprint arXiv:2303.07909 (2023).arxiv.org/abs/2303.07909
  160. [160]Lili Zhang, Xi Liao, Zaijia Yang, Baihang Gao, Chunjie Wang, Qiuling Yang, and Deshun Li. 2024. Partiality and misconception: Investigating cultural representativeness in text-to-image models. In Proceedings of the 2024 CHI conference on Human factors in computing systems. 1–25.
  161. [161]Sixian Zhang, Bohan Wang, Junqiang Wu, Yan Li, Tingting Gao, Di Zhang, and Zhongyuan Wang. 2024. Learning multi-dimensional human preference for text-to-image generation. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 8018–8027.
  162. [162]Shu Zhang, Xinyi Yang, Yihao Feng, Can Qin, Chia-Chih Chen, Ning Yu, Zeyuan Chen, Huan Wang, Silvio Savarese, Stefano Ermon, Caiming Xiong, and Ran Xu. 2024. HIVE: Harnessing Human Feedback for Instructional Visual Editing. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  163. [163]Ziyi Zhang, Sheng Li, Chen Shi, Yibing Zhang, Yong Luo, Han Hu, Bo Du, Yonggang Wen, and Dacheng Tao. 2025. Aligning Text-to-Image Diffusion Models With Constrained Reinforcement Learning. IEEE Transactions on Pattern Analysis and Machine Intelligence (2025).
  164. [164]Enmin Zhao, Renye Yan, Jinqiu Li, Kai Li, and Junliang Xing. 2022. Alpholdem: High-performance artificial intelligence for heads-up no-limit poker via end-to-end reinforcement learning. In AAAI Conference on Artificial Intelligence, Vol. 36, 4689–4697.
  165. [165]Ruoyu Zhao, Mingrui Zhu, Shiyin Dong, De Cheng, Nannan Wang, and Xinbao Gao. 2025. CatVersion: Concatenating Embeddings for Diffusion-Based Text-to-Image Personalization. IEEE Transactions on Circuits and Systems for Video Technology (2025).
  166. [166]Zeyu Zhao, Connor Rees, Gavin Bailey, Matt Jones, Simon Robinson, and Jennifer Pearson. 2026. Less Redraw, More Explore: Suggestion and Completion for Sketch-to-Image. In Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems. 1–11.
  167. [167]Jialun Zhong, Wei Shen, Yanxue Li, Songyang Gao, Hua Lu, Yicheng Chen, Yang Zhang, Wei Zhou, Jinjie Gu, and Lei Zou. 2025. A comprehensive survey of reward models: Taxonomy, applications, challenges, and future. arXiv preprint arXiv:2504.12328 (2025).arxiv.org/abs/2504.12328
  168. [168]Chao Zhou, Tianyi Wei, and Nenghai Yu. 2025. Scale Your Instructions: Enhance the Instruction-Following Fidelity of Unified Image Generation Model by Self-Adaptive Attention Scaling. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV).
  169. [169]Jiayi Zhou, Renzhong Li, Junxiu Tang, Tan Tang, Haotian Li, Weiwei Cui, and Yingcai Wu. 2024. Understanding nonlinear collaboration between human and AI agents: A co-design framework for creative design. In Proceedings of the 2024 CHI conference on human factors in computing systems. 1–16.
  170. [170]Jingyuan Zhu, Huimin Ma, Jiansheng Chen, and Jian Yuan. 2025. Isolated Diffusion: Optimizing Multi-Concept Text-to-Image Generation Training-Free With Isolated Diffusion Guidance. IEEE Transactions on Visualization and Computer Graphics (2025).

Paper details

vlnc:W_8zw4jpaq
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC0-1.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents