TraceMind: Predicting User Information Uptake from Low-Cost Interaction Traces during Human-LLM Content Co-Generation

CCS Concepts

• Human-centered computing → User models. keywords: Information Uptake; AI-Assisted Writing; Large Language Models; User modeling; Interaction traces

Introduction

In human-LLM content co-generation, such as drafting an announcement or recruitment post, AI-generated information can enter final artifacts without being adequately processed by the user [58, 90]. Overlooked information can lead to consequential errors or miscoordination, particularly in public settings such as journalism, public communication, or online posting [21, 37, 69]. Avoiding such oversight requires modeling what users actually take up during human-LLM co-generation.

Prior work often focuses on how AI-generated content is adopted into final artifacts [35, 90], offering limited insight into which information users have actually taken up. Information uptake can be considered at multiple levels, including recognition of information, comprehension of its meaning, and application in subsequent decisions or actions [29, 45]. In this work, we focus on recognition-level information uptake, operationalized as whether a user can correctly recognize what an atomic information unit in the final co-generated draft states immediately after completing the task [19, 20]. Following prior work on proposition segmentation and atomic evaluation, we define an atomic information unit as a self-contained statement that expresses a single piece of information [32, 65].

In addition, prior work on modeling users’ information-processing behavior has often relied on gaze tracking or other specialized sensing [11, 34], which can be difficult to deploy in everyday settings. In contrast, lower-burden mouse and keyboard interaction traces are readily available in everyday browser and editor interactions [25, 28]. However, whether such traces provide sufficient evidence of information uptake remains unclear.

In this paper, we ask whether such uptake can be assessed at the level of atomic information units in open-ended co-generation and predicted from low-cost interaction traces produced during the process.

Answering this question requires addressing two challenges. First, the open-ended uptake assessment challenge: unlike prior interaction-trace studies that assess predefined structural units (e.g., documents [28], paragraphs [78], individual HTML elements [26], or predefined content blocks [98]), the information units in open-ended co-generation emerge dynamically and cannot be fixed as assessment targets in advance. Second, the interaction-trace alignment challenge: interaction events cannot be directly mapped to atomic information units, as the same information may move across Chat and Draft, change through revision, and be rendered differently over time.

To address the open-ended uptake assessment challenge, we collected data from 62 participants across three everyday co-generation tasks. For each final draft, we extracted atomic information units, retained those suitable for recognition assessment, and generated a post-task recognition question for each retained unit. Participants’ answers provided performance-based uptake labels, yielding a dataset of 1187 labeled atomic information units paired with Chat and Draft histories, interaction events, and layout snapshots.

To address the interaction-trace alignment challenge, we present TraceMind, a framework that recovers and aligns the fragmented interaction history of each atomic information unit and uses the resulting traces to predict information uptake. TraceMind first identifies each atomic information unit’s semantic occurrences across Chat and Draft histories by locating spans that express the same information. It then recovers the on-screen layouts corresponding to these occurrences and identifies the interaction records associated with them over time. Finally, TraceMind models the unit-aligned traces as complementary spatial, temporal, and workflow-informed evidence to predict uptake.

Evaluated on the collected dataset, TraceMind achieved 81.17% AUROC, 80.58% balanced accuracy, 49.34% AUPRC$_{\mathrm{non}}$, and 73.05% macro-F1, outperforming all learned baselines across all four metrics. Our findings suggest that information uptake is an unfolding process that cannot be captured by isolated interaction signals alone, and that sustained active engagement (e.g., revisiting, revising, and cross-surface checking) provides particularly informative evidence for modeling uptake.

The contributions of this work are as follows:

• A performance-based method for assessing recognition-level information uptake in open-ended co-generation, using dynamically derived atomic information units as assessment targets.

• TraceMind, a framework for predicting information uptake from low-cost interaction traces, by aligning users’ evolving interactions with atomic information units across the co-generation process.

• Empirical insights into information uptake as an unfolding process, highlighting sustained active engagement as particularly informative for modeling uptake. Our findings motivate the design of uptake-aware human-LLM co-generation systems.

Related Work

Our work draws on three areas of HCI research. First, we review human-LLM content co-generation as a broad HCI task context and highlight our shift from prior work’s focus on the co-generated content itself to what users actually take up from that content. Second, we review subjective and performance-based approaches to user-state assessment, highlighting the challenge of extending performance-based measurement to open-ended tasks. Finally, we review interaction-trace-based user modeling, focusing on the opportunities and limitations of low-cost traces for modeling how users engage with evolving information.

Human-LLM Content Co-Generation and Information Uptake

Human-LLM content co-generation has become common across a wide range of HCI domains, including creative writing [12, 93], educational writing [38], professional communication [21], and personal journaling [41, 97]. Across these settings, LLMs can support users in diverse roles, such as generating ideas, drafting text, and revising content [23, 36, 52]. Prior work has also examined how users collaborate with LLMs during this process, such as how they accept suggestions, edit generated text, and divide writing work with AI [6, 16, 67].

Recent studies have raised concerns about how users engage with AI-generated content during human-LLM co-generation. Work on content adoption examines whether and how AI-generated material enters the final artifact [35, 52] and shows that users may substantially reuse AI suggestions [1]. Recent work also examines users’ overreliance on AI, showing that users may inappropriately accept incorrect AI outputs [7, 86]. Studies have further found that users may copy or accept AI-generated content with limited scrutiny [6, 58], while AI assistance can shape users’ final expressions or opinions [31, 37, 99]. Together, this work highlights the risks of excessive reliance on AI-generated content, but largely focuses on the co-generated content itself—what users adopt, rely on, or ultimately produce.

We instead focus on what users themselves take up from co-generated content. Prior work has begun to reveal consequences of AI assistance for users’ memory and awareness, including reduced recall of what users wrote [44] and confusion about whether content originated from themselves or AI [99]. Building on these observations, we seek to measure and predict recognition-level information uptake, operationalized as whether a user can correctly recognize what an atomic information unit in the final co-generated draft states immediately after completing the task [19, 20]. Our goal is to model which information users have actually taken up in co-generated content, enabling future co-generation systems to identify content that may require additional user attention or review.

Assessing User States in HCI Tasks

Subjective measures are widely used in HCI to assess user states such as perceived learning, confidence, trust, workload, and engagement. For example, search-as-learning studies collect perceived learning outcomes [13], AI-assisted decision-making studies use self-reported confidence to examine reliance and calibration [61], and recent work measures perceived self-regulated learning during information seeking [85]. Workload is also commonly assessed through subjective scales such as NASA-TLX [5, 51]. These measures are easy to collect, but they may not align with behavioral or performance outcomes and can be affected by retrospective recall and individual calibration [5, 61, 85]. They also require users to explicitly report their state, which can add burden and interrupt natural interaction [40].

Performance-based assessment instead asks users to demonstrate what they recognize, recall, or understand after interaction. For example, search-as-learning research uses pre- and post-task knowledge tests [13, 22, 84]; health communication studies use recognition and recall tests [19, 71]; and visualization research evaluates recall and comprehension of presented information [42, 94]. These approaches provide directly scored outcomes rather than relying only on users’ own judgments. Among these measures, recognition is commonly assessed through statement recognition or multiple-choice questions [19]. More broadly, automatic question-generation research has shown that assessment questions can be generated from source text [46], and recent work has explored LLM-generated multiple-choice questions for scalable assessment [30, 47]. Together, these studies provide a foundation for constructing performance-based recognition assessments from textual content.

However, many performance-based assessments rely on fixed, predefined assessment units. For example, search studies define topics and knowledge prompts before the task [13, 22]; health and visualization studies assess users on fixed texts or datasets [19, 42]; and related interaction studies organize measurement around predefined documents, paragraphs, or interface elements [26, 28, 78]. Open-ended human-LLM co-generation is different: each user produces a different artifact, and its information units emerge during interaction. Existing co-writing studies therefore tend to evaluate content adoption, artifact-level outcomes, or user perceptions [6, 35, 52], rather than performance on the specific information contained in each user’s final artifact. Our work addresses this gap by extending performance-based uptake assessment to open-ended co-generation.

Modeling User States from Interaction Traces

Interaction traces have long been used as indirect evidence of user states, such as attention, engagement, and information processing. In web search, dwell time, scrolling, and cursor behavior have been used to estimate document relevance and user interest [28, 33]. In online reading, viewport exposure, reading depth, and backtracking have supported models of attention and engagement [26, 80]. Related work has also used interaction behavior to predict session-level knowledge gain [?] and to characterize AI reliance and over-reliance [58, 60].

A major line of this work relies on gaze and other specialized sensing. Gaze-based studies use fixations, refixations, regressions, and reading paths to model reading behavior and user attention [11, 27]. Such signals provide rich spatial evidence, but introduce practical constraints: dedicated eye trackers, wearable devices, or continuous camera-based sensing can be difficult to deploy in everyday settings and may raise privacy concerns [34, 89]. These constraints motivate greater use of interaction traces that are already available in ordinary interfaces.

Lower-cost approaches rely on browser- and editor-level traces such as dwell time, viewport exposure, scrolling, cursor movement, text selection, copying, and editing [25, 48]. Because these traces are naturally produced during routine interaction, they provide opportunities for modeling user states in everyday interfaces [25, 28]. These traces have been used to identify engagement patterns [25], estimate document relevance [28], predict knowledge gain [?], and characterize behavioral indicators of AI overreliance [58].

However, low-cost interaction traces remain indirect and ambiguous proxies for user states [25, 56]. Moreover, prior work often summarizes such traces within fixed analysis scopes, such as pages, documents, sessions, or task instances [25, 28, 61, ?]. This makes such approaches difficult to apply directly to open-ended co-generation. We instead follow how users’ interactions with the same piece of information evolve over time, enabling information uptake to be modeled across the co-generation process.

Data Collection Study

To obtain fine-grained uptake labels in open-ended co-generation, we conducted a controlled data collection study. Our platform dynamically derived atomic information units from each final draft and generated corresponding post-task recognition questions. Participants’ responses yielded unit-level uptake labels, which were paired with the interaction traces recorded throughout co-generation to form the resulting dataset. We next describe the study design, data collection system, and resulting dataset.

Study Design

Participants. We recruited 62 participants through social media (30 identified as female and 32 as male; aged 18–30 years, $M = 21.48$, $SD = 1.95$). Participant demographics are provided in Appendix A.1. On 5-point Likert scales, participants’ self-reported familiarity with conversational LLMs averaged 4.10 ($SD = 0.43$), and their frequency of LLM use over the past month averaged 4.19 ($SD = 0.84$; 1 = almost never and 5 = multiple times daily). For writing-related tasks (e.g., drafting announcements, essays, or reports) specifically, participants reported both the frequency (1 = never, 5 = always) and the degree of reliance (1 = entirely self-written, 5 = almost entirely LLM-dependent) on LLMs, averaging 4.23 ($SD = 0.68$) and 3.89 ($SD = 0.84$), respectively. Each participant received $8 as compensation.

Task Design. We designed three co-generation tasks: a club team-building announcement, a roommate recruitment post, and a lecture announcement. Each participant was randomly assigned to one task. Table 1 summarizes the overall task settings; Appendix A.2 provides the complete participant-facing task instructions.

Task	Settings
Club team-building announcement	Organize an annual club team-building trip to Historic Town H, including coach transportation, visits to several local attractions, and dinner at Restaurant R. Draft an announcement introducing the activity purpose, itinerary, venues, transportation, attire, local cuisine, preparation, and other useful information.
Roommate recruitment post	Recruit one roommate for a D2–4×4 unit at Apartment A during a two-month summer program at University B. The participant and two other students have already confirmed their stay. Draft a public post introducing the apartment, amenities, neighborhood, transportation, move-in arrangements, current roommates, and expectations for shared living.
Lecture announcement	The university will host an AI lecture at 2:00 p.m. the following Friday, featuring three academicians from University A—Scholar P1, Scholar P2, and Scholar P3. Each scholar will give an academic talk followed by a Q&A session. Draft an announcement introducing the speakers, their research areas and representative work, the event theme and value, suggested preparation, and other useful information.

We selected the tasks based on four considerations. First, they reflect familiar everyday scenarios in which people plausibly use LLMs for writing, increasing ecological validity. Second, unnoticed errors in the resulting public-facing final draft could lead to serious real-world consequences, underscoring the importance of users’ information uptake. Third, each prompt allowed substantial flexibility in content creation, enabling diverse co-generation behaviors. Fourth, each deliverable contained multiple discrete details (e.g., schedules, locations, rules, profiles) that could be assessed through information-unit-level recognition questions.

Procedure. Participants first provided informed consent and completed a pre-task questionnaire. They then reviewed a study guide introducing the co-generation task, the TraceMind system, and the study procedure. Participants accessed our publicly hosted web-based study platform directly through their browsers to complete the co-generation task (see Section 3.2.1). To ensure reliable capture of mouse interaction traces, they were required to complete the study on a laptop using an external mouse. To ensure sufficient interaction and analyzable content, participants were required to co-generate a draft of at least 1500 characters. After completing the task, they uploaded their screen recordings. Each session lasted approximately 30 minutes.

Data Collection System

User Interface. The study platform comprised two work surfaces (see Figure 2). During the co-generation task (fig:2a), participants worked in a side-by-side workspace with the LLM chat on the left and an editable Draft panel on the right. Participants could compose prompts by typing or using voice dictation. They could copy content from LLM responses into the draft or write and revise the draft directly. The Chat surface used GPT-5.4 as the underlying LLM.

After submitting their final drafts, participants proceeded to the post-task recognition interface (fig:2b). The interface presented four-option recognition questions based on the final draft. For each question, participants selected an answer and rated their confidence on a five-point scale before proceeding. Questions were presented one at a time without backward navigation to preserve item independence and limit cross-item cueing. We use answer correctness as the primary outcome label for information uptake. As a complementary analysis, we examine how self-reported confidence relates to recognition correctness and interaction traces in Section 6.3.

Interaction Event Logging. We instrumented the web-based study platform with JavaScript event listeners to record participants’ interaction behaviors during co-generation. The logger sampled timestamped pointer coordinates ($x,y$) at approximately 10 Hz and recorded user-initiated operations in both the Chat and Draft surfaces. As summarized in Table 2, we distinguish user behaviors captured consistently across both surfaces from those specific to the Chat or Draft surface.

Type	Event	Description
Shared across Chat and Draft surfaces	pointer_move	The user moves the pointer over content in either surface.
Shared across Chat and Draft surfaces	pointer_click	The user clicks content or an interactive element.
Shared across Chat and Draft surfaces	scroll_change	The user scrolls the Chat or Draft surface.
Shared across Chat and Draft surfaces	text_selection_episode	The user selects a text span in a chat message or the Draft.
Shared across Chat and Draft surfaces	clipboard_copy	The user copies selected text from either surface.
Shared across Chat and Draft surfaces	clipboard_cut	The user cuts selected text from an editable field in either surface.
Shared across Chat and Draft surfaces	clipboard_paste	The user pastes text into the chat input or Draft.
Shared across Chat and Draft surfaces	focus_in/out	The user moves the interaction focus into or out of a surface element.
Shared across Chat and Draft surfaces	activity_idle_start/end	The user remains inactive for at least 30 seconds and subsequently resumes interaction.
Chat surface-specific	chat_submit	The user submits a prompt composed through keyboard, voice, or mixed input. Keyboard interactions within the same prompt are represented collectively at submission.
Chat surface-specific	voice_input_start	The user activates the microphone and begins composing a prompt through voice input.
Chat surface-specific	voice_input_stop	The user stops voice input manually or reaches the recording time limit.
Draft surface-specific	text insertion	The user inserts or revises draft text through keyboard or input-method input, including typing over selected text.
Draft surface-specific	text deletion	The user deletes draft text using Backspace, Delete, or an equivalent editing operation without inserting replacement text.
Draft surface-specific	final_submit	The user submits the completed draft.

To align interaction events with the content displayed on screen, we recorded change-triggered snapshots of the Chat and Draft surfaces. When the surface content changed (e.g., when an LLM response appeared or the draft was edited), the logger captured paired content and layout snapshots, including the full surface text, block-level offsets, and line- and grapheme-level bounding boxes. When the visible viewport changed (e.g., during scrolling), the logger captured visibility snapshots containing the scroll position, viewport bounds, visible text lines, and the visible proportion of each line.

Together, these records preserve the Draft history, Chat history, interaction event log, and layout and visibility snapshots used as session record inputs to TraceMind (Figure 4).

Post-task Question Generation

After participants submitted their final drafts, the system automatically generated post-task recognition questions to assess participants’ immediate information uptake (Figure 3).

The system first divided each final draft into text segments and extracted candidate atomic information units using criteria adapted from prior work on proposition segmentation [32]. Each unit had to (1) be well-formed, (2) be self-contained, (3) express a single piece of information, and (4) be supported by the draft. The system then removed semantically equivalent units (e.g., “Check-in is scheduled for 7:30-8:00 a.m.” and “The check-in window is 7:30-8:00 a.m.”). Units likely to yield overly generic answers were identified and assigned lower priority during question sampling. For example, “a suitable roommate” in “We hope to find a suitable roommate soon” provides no specific roommate characteristics and therefore makes it difficult to generate discriminative distractors.

For each retained atomic information unit, the system generated a question stem and the correct answer, then generated three distractors using the complete final draft as context. Distractor generation followed five criteria: (1) same dimension: match the correct answer’s semantic type and specificity; (2) plausible alternatives: be credible in the scenario and not easily ruled out by common knowledge; (3) non-overlapping: remain semantically distinct from all other options, without containment, overlapping ranges, or shared endpoints; (4) matched form: use the same language and comparable length, grammar, and formatting to avoid answer cues; and (5) unsupported by the final draft: no passage in the final draft may support the distractor as a valid answer to the question. For example, a roommate recruitment post might list a gym, study area, package collection service, and laundry room as apartment amenities. For a question asking which amenity is listed, “a laundry room” cannot serve as a distractor for “a gym”, because both are valid answers.

Each resulting four-option, single-choice question item was validated for (1) clear and unambiguous wording, (2) a unique correct answer directly supported by the final draft, (3) comparable and non-overlapping options, and (4) dependence on the final draft rather than common knowledge. Invalid items underwent one targeted repair attempt and were discarded if they still failed validation.

If the qualified question pool exceeded $K_{\max} = 20$, the system performed stratified sampling based on the locations of the source text supporting each answer. Finally, given that LLM-generated questions can exhibit biases in the positions assigned to correct answers [83], the system then randomly shuffled the answer options to mitigate positional bias.

We used GPT-5.4 for information-unit extraction, question generation, and question validation throughout this pipeline.

Dataset Construction

We constructed the dataset from 62 participants, each contributing one completed study session. For each session, the archived data

included the Chat and Draft histories, interaction-event logs, surface and visibility snapshots, recognition questions and responses, extracted information units, and their corresponding occurrences and exposure intervals.

The 62 sessions included 1187 information units with valid labels based on post-task recognition responses. These included 428 units from the club team-building announcement task, 423 from the roommate recruitment post task, and 336 from the lecture announcement task. Participants correctly recognized 881 units and incorrectly recognized 306, corresponding to an overall recognition accuracy of 74.22%. On average, participants spent 21.64 minutes in the writing interaction phase ($SD = 7.03$) and 27.95 minutes in the full session including the post-task recognition assessment ($SD = 11.39$); each session contained 17.19 chat messages, 33.94 draft versions, and 3525.79 interaction events. Each session also contained dense surface-state logs, with an average of 8245.27 snapshots capturing changes in surface content, rendered layout, and visibility.

TraceMind Framework

In this section, we present the TraceMind framework (see Figure 4). TraceMind takes as input a target information unit from the final draft and records from the corresponding user–LLM co-generation session. These session records include the Draft history, Chat history, interaction event log, and layout and visibility snapshots. For each target information unit, TraceMind predicts whether the user took up that information.

The key idea of TraceMind is to organize fragmented cross-interface interaction traces around where and how each information unit appeared over time. For each unit, TraceMind first identifies its semantic occurrences across the Chat and Draft histories (Section 4.1) and reconstructs layout-aware occurrence states that capture how these occurrences were rendered and exposed over time (Section 4.2). Next, it aligns and encodes the associated interaction traces into spatial, temporal, and workflow-informed representations (Section 4.3). Finally, TraceMind models these representations in three separate branches and combines their predictions into the final uptake prediction.

Identifying Semantic Occurrences of Each Information Unit

Throughout the co-generation process, the same information may first appear in the Chat, later be transferred to the Draft, and then be revised as the draft evolves. Therefore, given a target information unit in the final draft, TraceMind first identifies corresponding spans across the Draft and Chat histories that express the same information. We refer to each identified span as a semantic occurrence of the target information unit. As shown in Step 1 of Figure 4, TraceMind identifies these occurrences in two steps: Draft history back-tracing and Chat history matching.

Draft history back-tracing. We begin with the supporting span of the target unit in the final draft, associated with the unit’s post-task recognition question (Section 3.2.3). This span anchors the target unit to its occurrence in the final draft. Using text correspondence and recorded provenance, TraceMind traces the span backward through the Draft history to recover earlier spans in which the same information was retained or revised. Each recovered span is linked to the Draft version in which it appeared.

Chat history matching. TraceMind next searches the Chat history for spans corresponding to the recovered Draft occurrences.

It first accepts deterministic matches supported by exact text correspondence or recorded copy-and-paste provenance. For candidates that cannot be resolved deterministically, an LLM judge determines whether a Chat span expresses the same information despite different wording and, if so, returns the matched span. We use GPT-5.4 as the LLM judge. Together, the accepted Draft and Chat spans form the semantic occurrences of the target information unit, which are then passed to layout-aware state reconstruction (Section 4.2).

Reconstructing Layout-Aware Occurrence States for Each Semantic Occurrence

The same semantic occurrence may be rendered under different spatial conditions as the Chat and Draft evolve and the user scrolls. Because spatial interaction traces are meaningful only relative to the layout in which they were recorded, TraceMind represents each distinct rendering condition of a semantic occurrence as an occurrence state.

As shown in Step 2 of Figure 4, TraceMind traverses the layout and visibility snapshots and retrieves every recorded layout in which the semantic occurrence appears. We refer to these occurrence-specific layout records as layout observations. For each observation, TraceMind locates the occurrence’s text span in the recorded layout and recovers its rendered geometry. The observation also retains the corresponding Chat or Draft scope, interface surface and page instance, text content, and visibility condition.

TraceMind reconstructs occurrence states by grouping layout observations that provide equivalent spatial anchors. Observations are assigned to the same state only when they refer to the same semantic occurrence and preserve the same interface context, text content, visibility condition, and approximately the same bounding-box shape. As illustrated in Figure 5, layouts that differ only because scrolling moves the occurrence within the viewport are merged. In contrast, text reflow changes the bounding-box shape and therefore produces a new state because it alters the spatial relationship between the target text and nearby interaction traces. Changes in content, interface context, or visibility likewise produce separate states.

For each state with observable geometry, TraceMind retains a hard bounding box that tightly encloses the rendered target text. It also derives a soft bounding box by expanding the hard box by a radius $\rho$ on all sides. The hard box anchors the occurrence itself, whereas the soft box preserves nearby pointer activity that may reflect attention without entering the exact text region. These state-specific boxes support the spatial alignment described in Section 4.3.1.

Aligning and Encoding Interaction Traces

As shown in Step 3 of Figure 4, TraceMind relates the logged pointer and operation events to the target unit’s semantic occurrences and reconstructed occurrence states. This produces two records: target-relative pointer trajectories for each occurrence state and a target-linked operation sequence spanning the session. TraceMind encodes these records into three complementary representations: state-specific pointer-trajectory heatmaps, a time-binned activity sequence, and a workflow-informed feature vector.

State-Specific Pointer-Trajectory Heatmaps

Pointer positions are meaningful only relative to the layout in which they were recorded. TraceMind therefore aligns pointer coordinates to the target and encodes the resulting trajectories separately for each occurrence state. For state $s$, the hard bounding box $B_s = (l_s, t_s, r_s, b_s)$ provides the local spatial anchor, with width $w_s = r_s - l_s$ and height $h_s = b_s - t_s$.

Requiring the pointer to enter the hard bounding box would omit nearby movements that may still be associated with the target. We instead use the soft bounding box defined in Section 4.2 and retain pointer points inside the region obtained by expanding $B_s$ by $\rho$ on all sides:

When $\rho= 0$, the soft and hard bounding boxes coincide. Increasing $\rho$ retains a broader target-relative interaction context. We compare the effect of $\rho$ in Section 5.

To obtain a fixed-size spatial representation, we rasterize the retained trajectory as a $128 \times128$ heatmap. Each retained pointer point $p_i = (x_i, y_i)$ is mapped to heatmap coordinates $\pi_\rho(p_i) = (\bar{x}_i, \bar{y}_i)$, where

Here, $W = H = 128$ denotes the spatial grid resolution. This transformation independently normalizes the horizontal and vertical extents of the expanded region to the heatmap frame.

Each heatmap contains six channels: density, dwell, path, slow dwell, target proximity, and speed. For the five point-based channels, we distribute each point’s channel-specific weight over nearby pixels with a Gaussian kernel. For the path channel, we rasterize the line between consecutive points. Let $E_s(\rho)$ contain consecutive point pairs from the same trajectory segment that are separated by at most three seconds. The value of channel $c$ at heatmap location $q$ is

where $G_{\sigma_c}$ is a Gaussian kernel centered at the mapped point and weighted by $\omega_{i,c}$, and $L_{\sigma_c}$ rasterizes the line between two consecutive mapped points.

Because repeated activity can produce highly skewed channel values, we further apply

which compresses large intensities while preserving zero-valued regions. The resulting $H_s \in\mathbb{R}^{6 \times128 \times128}$ represents the target-relative pointer trajectory for state $s$. Alongside each heatmap, TraceMind retains a 16-dimensional state summary covering interaction amount, target proximity, relative position, interface scope, etc. For each information unit, we retain up to 16 occurrence states and use a binary mask to distinguish observed states from padding.

Figure 6 illustrates how increasing the soft bounding box radius $\rho$ expands the retained spatial context across the six heatmap channels.

Time-Binned Activity Sequence

The interaction log contains both frequent pointer events and sparse operations such as selection, copying, pasting, and editing. To preserve their order without allowing high-frequency events to dominate, TraceMind represents the target-linked operations on a normalized session timeline.

TraceMind first constructs event–target relations. Text operations (e.g., select, copy, paste, text insertion) are linked to the target unit when their affected range uniquely overlaps one of its semantic occurrences. Pointer-based events (e.g., pointer move, pointer click) are linked through the available occurrence-state geometry at the event time to measure the continuous distance from the pointer to the rendered target text. It converts this distance into proximity signals at multiple spatial scales, assigning stronger target-related evidence to closer pointer events.

We divide each session into $T = 512$ equal-width bins and construct, for each information unit, a time-binned activity sequence $\mathbf{x}^{\mathrm{op}} \in\mathbb{R}^{512 \times44}$. The vector for bin $k$ is

Here, $\mathbf{s}_k \in\mathbb{R}^{20}$ summarizes the interaction context within the bin, including pointer movements, clicks, scrolling, editing, selection, clipboard operations, visibility changes, and surface transitions. $\mathbf{r}_k \in\mathbb{R}^{16}$ summarizes the constructed event–target relations, including text- and pointer-relation counts, Chat or Draft scope, source context, layout observability, and distance-based proximity. Finally, $\mathbf{a}_k \in\mathbb{R}^{8}$ captures bin-level conjunctions between activity types and target-related evidence, such as copying, editing, clicking, or scrolling in a time bin where such evidence is present. We apply $\log(1+x)$ to count-like values and use a binary mask to distinguish observed from empty bins.

Workflow-Informed Interaction Features

The time-binned sequence preserves local temporal progression, but does not directly represent sparse, high-level behavioral patterns that span distant events, such as revisiting a source after editing the Draft, or copy-and-paste-based text transfer. TraceMind therefore uses the same target-linked operation sequence to derive a workflow-informed feature vector that summarizes higher-level engagement with the target unit.

We developed these features through iterative inspection of 15 pilot co-generation session replays collected before the main data collection study. These sessions were not included in the 62-session dataset described in Section 3.3. Two HCI researchers independently reviewed how users interacted with individual information units across Chat and Draft, without access to post-task recognition outcomes. The researchers captured recurrent patterns of observable user engagement. We retained recurrent patterns that could be computed objectively from the interaction logs.

The resulting 40-dimensional feature set captures eight behavior families: Chat-side repeated inspection, Draft-side repeated inspection, Draft-side inspection after editing, Draft editing and revision, copy-and-paste-based transfer, cross-surface text selection, cross-surface coordination, and source checking before revision. These behavior families also broadly align with prior work that characterizes rereading, revisitation, and content manipulation as observable traces of information processing [53, 74]. The complete feature set is provided in Appendix B.1. For each information unit, these features form a vector $\mathbf{x}^{\mathrm{wf}} \in\mathbb{R}^{40}$. Count- and duration-based features are transformed using $\log(1+x)$ before modeling.

Information-Uptake Prediction Model

The three representations above have different structures and capture complementary interaction evidence. As shown in Step 4 of Figure 4, TraceMind models each representation in a separate branch and combines the three branch probabilities through weighted-logit late fusion. This maps the heterogeneous signals into a common prediction space, enabling simple and effective fusion.

Spatial trajectory branch. TraceMind first applies channelwise mean and maximum pooling across the valid state-specific trajectory heatmaps. Their concatenation forms a 12-channel image that preserves both recurring and salient spatial patterns. A residual CNN with squeeze-and-excitation gating encodes this image into a 128-dimensional representation. In parallel, TraceMind averages the 16-dimensional state summaries across valid states and projects the resulting vector to 48 dimensions. The two representations are concatenated and passed through a linear classification head to produce the spatial-branch uptake probability $p_{\mathrm{traj}}$. We train this branch with class-weighted binary cross-entropy and use light channel dropout and Gaussian perturbation for regularization.

Temporal activity branch. TraceMind first projects each bin of $X^{\mathrm{op}}$ into a 48-dimensional hidden space. A three-layer temporal convolutional network (TCN), with dilation rates 1, 2, and 4, models interaction patterns over multiple temporal scales. Mask-aware attentive pooling then aggregates the observed bins into a temporal embedding. We first pre-train the TCN encoder through fold-local mask-bin reconstruction: 18% of the observed bins are masked, and a temporary decoder reconstructs their original interaction features from the surrounding temporal context. After pre-training, we discard the decoder and jointly fine-tune the encoder, attentive-pooling layer, and linear classification head using class-weighted binary cross-entropy. At inference time, the trained model processes the complete unmasked sequence and outputs the temporal-branch uptake probability $p_{\mathrm{temp}}$.

Workflow-informed feature branch. TraceMind models the 40-dimensional workflow-informed feature vector using ElasticNet-regularized logistic regression. Count- and duration-based features are log-transformed, missing values are imputed with training-set medians, and all features are standardized using training-set statistics. The model outputs the workflow-branch uptake probability $p_{\mathrm{wf}}$.

TraceMind combines the three probabilities by taking a positive weighted sum of their log-odds:

where $\sigma$ is the sigmoid function, $\alpha_{\mathrm{traj}}, \alpha_{\mathrm{temp}}, \alpha_{\mathrm{wf}} > 0$, and $\alpha_{\mathrm{traj}} + \alpha_{\mathrm{temp}} + \alpha_{\mathrm{wf}} = 1$. TraceMind predicts uptake when $\hat{p} \ge t$ and non-uptake otherwise. The decision threshold $t$ is selected using only the training data and fixed before evaluation on the held-out test set.

Model Evaluation

In this section, we evaluate TraceMind using the dataset collected in Section 3. We next describe the evaluation setup, overall performance, and ablation results.

Experimental Setup

Evaluation Protocol

We evaluate TraceMind on the dataset collected in Section 3, containing 1187 labeled information units from 62 participants. We use 5-fold grouped cross-validation with participant ID as the grouping variable, ensuring that all units from the same participant remain in the same split and avoiding participant-level leakage. We aggregate predictions from the held-out folds for final evaluation.

Baseline Models

We compared TraceMind with the following baseline models.

• Content & Context + ElasticNet: Prior work shows that content and task context can affect interaction-based judgments independently of observed behavior [62]. This baseline combines a frozen embedding of the information-unit text with non-behavioral attributes, including text length, normalized position in the final draft, provenance, and task type. The resulting features are classified using ElasticNet-regularized logistic regression.

• Conventional Interaction + GBT: Conventional interaction statistics, such as exposure time, scrolling, and pointer activity, have been used to predict reading difficulty and content usefulness [24, 62, 98]. This baseline summarizes each information unit using its viewport exposure, pointer dwell and movement, clicks, scrolling, selection, copy, paste, and editing counts, and trains a Gradient Boosting Tree (GBT) classifier. It excludes the revisit, cross-surface, and source-checking relations of our workflow-informed feature set.

• Workflow-Informed Features + GBT: This baseline uses the 40-dimensional workflow-informed feature vector $\mathbf{x}^{\mathrm{wf}}$ in Table 6 as input to GBT.

• LSTM + DNN: Recurrent models have been widely used to encode mouse interaction sequences [4]. This baseline processes the time-binned interaction representation $\mathbf{X}^{\mathrm{op}} \in\mathbb{R}^{512 \times44}$ using three Long Short-Term Memory (LSTM) layers with 128, 64, and 32 hidden units. The resulting temporal representation is concatenated with workflow-informed features and classified by a Deep Neural Network (DNN).

• Transformer + DNN: Transformer encoders have also been used to model cursor trajectories for attention prediction [87]. This baseline applies a Transformer encoder with sinusoidal positional encodings to $\mathbf{X}^{\mathrm{op}}$ and reads out the temporal representation from a [CLS] token. The Transformer is trained from scratch, without the masked-bin pretraining used by TraceMind’s temporal branch. The other components are consistent with the baseline of LSTM + DNN.

• ResNet-50 + DNN: Image-based encodings paired with ResNet-50 provide a strong visual baseline for mouse-movement modeling [4, 87]. This baseline adapts the first convolutional layer of an ImageNet-pretrained ResNet-50 to the six-channel trajectory heatmaps described in Section 4.3.1, encodes each valid occurrence state, and mean-pools the state representations. The other components are consistent with the baseline of LSTM + DNN.

• Mouse2Vec + DNN: Mouse2Vec learns reusable mouse-behavior representations through self-supervised pretraining [96]. This baseline applies the frozen Mouse2Vec encoder to the raw pointer segments associated with an information unit and pools the resulting segment embeddings. The other components are consistent with the baseline of LSTM + DNN.

• Full-Input Early-Fusion DNN: This baseline receives the same spatial trajectory, temporal interaction, and workflow-informed inputs as TraceMind. It concatenates the spatial and temporal embeddings with the workflow-informed feature vector and predicts uptake using a DNN. The temporal encoder uses the same masked-bin pretraining as TraceMind.

Behavior type	Operationalized features
Chat-side repeated inspection	number of Chat visits, number of Chat revisits, number of delayed Chat revisits, binary indicator of delayed Chat revisit, maximum time gap between Chat revisits, normalized Chat visual revisit count per occurrence
Draft-side repeated inspection	number of Draft visits, number of Draft revisits, binary indicator of Draft revisit, number of delayed Draft revisits, binary indicator of delayed Draft revisit, mean time gap between Draft revisits, maximum time gap between Draft revisits, active Draft visit rate, normalized Draft visual revisit count per information-unit member
Draft-side inspection after editing	number of Draft revisits after editing, binary indicator of Draft revisit after editing
Draft editing and revision	number of manual edits, number of edit bouts, binary indicator of any edit, edits per Draft visit, normalized manually inserted text length
Copy-and-paste-based transfer	normalized Draft paste count per occurrence, normalized Draft paste count per information-unit member, normalized pasted text length per information-unit member, normalized Draft copy count per occurrence, normalized verified copy-paste chain count per occurrence, minimum latency from Chat copy to Draft paste
Cross-surface text selection	normalized Chat text selection count per occurrence, normalized Draft text selection count per occurrence
Cross-surface coordination	number of cross-surface switches, number of cross-surface engagement events, number of cross-surface bouts, number of distinct visited surfaces, number of distinct visited occurrences, number of cross-occurrence switches, number of event-linked opportunities
Source checking before revision	binary indicator of source check before revision, number of source checks before revision, minimum latency from source check to subsequent revision

Evaluation Metrics. Because our dataset contains significantly more uptake labels than non-uptake labels, conventional accuracy can overemphasize the majority class. We therefore report balanced accuracy and macro-F1 for thresholded predictions, which give equal importance to the two classes while capturing classwise recall and precision [9, 81]. We also report the area under the precision-recall curve for non-uptake $AUPRC_{\mathrm{non}}$. This metric evaluates the precision-recall tradeoff for detecting the less frequent non-uptake cases, which are particularly important for identifying information that may require additional review [15, 76]. We also report AUROC to measure the model’s overall ability to distinguish uptake from non-uptake across decision thresholds.

Overall Performance

As shown in Table 3, TraceMind achieved the strongest overall performance among the learned models across all four metrics. Compared with the strongest learned baseline for each metric, TraceMind improved AUROC by 4.49 percentage points, balanced accuracy by 12.38 points, $AUPRC_{\mathrm{non}}$ by 3.98 points, and macro-F1 by 8.21 points.

Model	AUROC	Bal. Acc.	$AUPRC_{\mathrm{non}}$	Macro-F1
Content & Context + ElasticNet	65.01	60.56	27.12	53.00
Conventional Interaction + GBT	62.33	61.75	32.05	60.25
Workflow-Informed Features + GBT	71.42	62.57	33.96	45.14
LSTM + DNN	75.58	65.84	44.01	64.84
Transformer + DNN	65.81	63.46	28.28	53.05
ResNet-50 + DNN	75.00	60.09	43.17	61.73
Mouse2Vec + DNN	72.58	68.20	37.20	62.76
Full-Input Early-Fusion DNN	76.68	58.62	45.36	59.88
TraceMind	81.17	80.58	49.34	73.05

The strongest competing baseline varied across metrics: Full-Input Early-Fusion DNN performed best on AUROC and $AUPRC_{\mathrm{non}}$, Mouse2Vec + DNN performed best on balanced accuracy, and LSTM + DNN performed best on macro-F1. As shown in Figure 7, TraceMind’s advantage was also sustained across decision thresholds.

Ablation Results

We next conduct a series of ablation analyses on TraceMind to examine the contributions of its three input branches and other key design choices.

Contribution of Three Input Branches

To assess the contribution of each input branch, we evaluate the performance of each branch individually and the overall model performance when each branch is removed in turn.

As shown in Table 4 (Input branches block), all three branches provided predictive signal independently, with AUROC ranging from 74.69% to 78.00%. Moreover, removing the temporal branch produced the largest decreases in AUROC and balanced accuracy (2.06 and 9.04 percentage points), removing the workflow-informed branch produced the largest decrease in macro-F1 (11.43 points),

Ablated Design	Model Variant	AUROC	Bal. Acc.	AUPRC$_{\mathrm{non}}$	Macro-F1
Input branches	Spatial only	77.91	69.69	43.96	58.61
Input branches	Temporal only	78.00	72.85	43.77	65.41
Input branches	Workflow only	74.69	72.14	40.32	63.18
Input branches	w/o Spatial	80.45	77.36	44.85	69.75
Input branches	w/o Temporal	79.11	71.54	47.61	63.64
Input branches	w/o Workflow	80.25	73.95	48.76	61.62
Interaction traces	Chat-only traces	75.49	67.94	39.84	56.30
Interaction traces	Draft-only traces	75.96	73.87	40.36	64.86
Alignment	w/o semantic alignment	72.91	70.36	33.21	65.09
Alignment	w/o layout-state alignment	80.70	78.01	45.25	70.70
TCN pretraining	Temporal only (w/o TCN pretraining)	76.07	69.76	36.01	65.43
TCN pretraining	TraceMind (w/o TCN pretraining)	79.94	72.27	43.60	66.51
—	TraceMind	81.17	80.58	49.34	73.05

and removing the spatial branch produced the largest decrease in $\mathrm{AUPRC}_{\mathrm{non}}$ (4.49 points). These results suggest that the three branches capture distinct and complementary aspects of information uptake.

5.3.2 Contribution of Chat and Draft Interaction Traces. To examine how interactions across the two interfaces contribute to prediction, we restricted all three branches to either Chat-only or Draft-only traces, and omitted cross-surface workflow features. All model components and training settings were kept unchanged.

As shown in Table 4 (Interaction traces block), Draft-only traces retained more predictive signal than Chat-only traces. Moreover, combining interaction traces from both interfaces consistently outperformed using either interface alone. Compared with using Chat-only and Draft-only traces, TraceMind improved AUROC by 5.68 and 5.21 points, balanced accuracy by 12.64 and 6.71 points,

AUPRC$_{\mathrm{non}}$ by 9.50 and 8.98 points, and macro-F1 by 16.75 and 8.19 points, respectively. These results suggest that Draft interactions provide the stronger individual signal, while Chat interactions contribute complementary evidence to the full model.

Contribution of Semantic and Layout-state Alignment

5.3.3 Contribution of Semantic and Layout-state Alignment. To examine the contribution of the two alignment stages, we evaluated two ablated versions of TraceMind. For the semantic-alignment ablation, we rebuilt each information unit’s unit-linked inputs solely from its final-draft occurrence, excluding matched occurrences in Chat and earlier Draft versions, while retaining the temporal branch’s session-level context. For the layout-state ablation, we retained all identified semantic occurrences and the bbox-anchored trajectory encoding, but merged the layout states belonging to each occurrence into one occurrence-level trajectory image. All other model components and training settings were kept unchanged.

As shown in Table 4 (Alignment block), using only the final-draft anchor reduced AUROC by 8.26 percentage points, balanced accuracy by 10.22 points, AUPRC$_{\mathrm{non}}$ by 16.13 points, and macro-F1 by 7.96 points. Removing layout-state alignment had a smaller effect after fusion, but still reduced balanced accuracy by 2.57 points, AUPRC$_{\mathrm{non}}$ by 4.09 points, and macro-F1 by 2.35 points. Its effect was more pronounced within the spatial trajectory branch, whose AUROC decreased from 77.91% to 74.12%. Together, these results suggest that recovering an information unit’s semantic occurrences across Chat and Draft provides substantial predictive signal beyond its final-draft occurrence alone, while layout-state alignment preserves complementary spatial evidence that the other branches can partially compensate for.

5.3.4 Contribution of TCN Pretraining on Temporal Patterns Branch. To isolate the contribution of TCN pretraining, we removed the masked-bin-reconstruction pretraining stage and trained the temporal TCN from scratch. All other model components and training settings were kept unchanged.

As shown in Table 4 (TCN pretraining block), pretraining improved the temporal branch’s AUROC by 1.93 percentage points and AUPRC$_{\mathrm{non}}$ by 7.76 points. Its benefit persisted after fusion: compared with training the TCN from scratch, the full TraceMind model improved AUROC by 1.23 points, balanced accuracy by 8.31 points, AUPRC$_{\mathrm{non}}$ by 5.74 points, and macro-F1 by 6.54 points. These results suggest that masked-bin-reconstruction pretraining helps the TCN learn temporal representations that complement the other two branches.

5.3.5 Effect of Soft Boundary. To examine how much spatial context should be retained around each rendered occurrence, we varied the bbox-anchored soft-boundary radius $\rho$ and rebuilt the spatial trajectory input for each setting. All other model components and training settings were kept unchanged.

As shown in Figure 8, restricting trajectories to the target bounding box ($\rho= 0$) substantially weakened the spatial branch, whose AUROC decreased to 52.77%. Performance improved as nearby movements were included, with $\rho= 250$ yielding the strongest overall fusion results. Expanding the boundary further did not consistently improve the fused model. These results suggest that pointer movements near an information unit provide useful evidence beyond direct entry into its bounding box, while overly broad regions add limited additional signal.

Behavioral Analysis

Beyond evaluating TraceMind’s predictive performance, we further conduct a behavioral analysis on the collected data to examine user behavioral patterns associated with information uptake during human–LLM co-generation.

Information Uptake Across Exposure Duration and Active Engagement

In this section, we examine how exposure duration and active engagement were associated with information uptake. We first examined whether greater exposure duration and individual active-engagement signals were associated with higher information uptake. For signals with sufficiently many positive observations (e.g., visible duration and target-near dwell), we divided units into low and high groups at the median. We then estimated task- and sample-kind-adjusted uptake probabilities using logistic standardization,

with 95% CIs obtained from 1,000 participant-cluster bootstrap samples. As shown in Figure 9, uptake was significantly higher in the high-exposure group than in the low-exposure group ($p < 0.05$). Higher active engagement was also associated with greater uptake, with significant differences for target-near dwell ($p < 0.01$), clicks ($p < 0.01$), revisits ($p < 0.001$), and manual edits ($p < 0.01$). Overall, both greater exposure and more active engagement were positively associated with information uptake, with stronger evidence for the active-engagement signals.

We then examined the joint pattern of exposure duration and active engagement. We constructed an equal-weight active-engagement index from six standardized components: target-near clicking, revisiting, occurrence inspection, Chat–Draft coordination, Draft revision, and selection and transfer. We divided all units at the global medians of visible duration and the engagement index, retaining every unit in one of four groups: Low Exposure Low Engagement, Low Exposure High Engagement, High Exposure Low Engagement, and High Exposure High Engagement. As shown in Figure 9, visible duration and the engagement index showed a moderate overall positive correlation ($r_s = .61$), with the concordant groups (High Exposure High Engagement and Low Exposure Low Engagement) comprising 72.2% of units and the discordant groups (High Exposure Low Engagement and Low Exposure High Engagement) comprising 27.8%.

We further compared information uptake across the four groups. We estimated uptake probabilities using task-adjusted logistic standardization, with 95% CIs obtained from 1,000 participant-cluster bootstrap samples. We then compared High and Low Engagement within each exposure level (Figure 9C). High Engagement showed higher estimated uptake than Low Engagement under both low exposure duration (71.5% vs. 67.0%) and high exposure duration (81.8% vs. 75.3%). Participant-clustered GEE tests with Holm correction showed that this increase was significant within the high-exposure group ($p < 0.01$), but not within the low-exposure group.

Within the high-exposure group, we further examined which engagement behaviors distinguished High and Low Engagement units, whose uptake probabilities differed significantly (Figure 9D). The largest adjusted standardized differences were observed in revisiting (1.31SD), Chat–Draft coordination (1.24SD), Draft revision (1.10SD), and occurrence inspection (1.05SD).

Together, exposure duration and active engagement were each positively associated with information uptake and were also positively associated with each other. Estimated uptake was highest when both were high. Among highly exposed units, more sustained and coordinated engagement was associated with higher uptake.

Information Uptake Across Content Adoption Completeness and Post-Adoption Engagement

In this section, we examine whether information uptake varied with how LLM-provided content entered the Draft and how users subsequently engaged with it. We first categorized each valid information unit based on whether and how it was adopted from Chat into the Draft. For each unit, we used an LLM judge (GPT-5.4-mini) to determine whether its first Draft occurrence was semantically matched to an earlier assistant message. Units with no such match were classified as non-adoption. For matched units, we further distinguished complete adoption, where the normalized Draft occurrence appeared exactly as written in the matched Chat message, from incomplete adoption, where the same information was transferred with rephrasing. Given the small number of non-adoption units (13 units from nine participants, all showing uptake), our subsequent analysis only focuses on the Chat-adopted units, including complete and incomplete adoption units.

To begin with, complete and incomplete adoption showed nearly identical uptake probabilities (Figure 10A): 73.7% for complete adoption and 74.5% for incomplete adoption. Thus, preserving the assistant’s exact wording alone was not significantly associated with uptake.

Post-adoption Draft engagement showed a clearer association with uptake (Figure 10B). The overall effect of editing depth was significant ($p < 0.01$). Among Chat-adopted units, uptake increased from 67.7% with no target edit to 76.3% after a single edit bout and 79.4% after repeated revision.

For the timing analyses, we estimated standardized logistic predictions over the normalized time of first Draft appearance (Figure 10C). The timing of initial insertion did not show a reliable association with uptake. By contrast, the timing of subsequent engagement showed a clearer association (Figure 10D). Among units with later re-engagement, we classified a unit as receiving a final-stage review when its last target-related interaction occurred within the final 10% of the task or within 120 seconds before final draft submission; all other cases were classified as earlier re-engagement. Units receiving a final-stage review had a higher uptake probability than those whose re-engagement occurred earlier ($p < 0.05$).

Together, these results suggest that uptake was associated less with how or when information entered the Draft than with how it was subsequently processed. Specifically, exact wording and initial timing did not reliably distinguish uptake, whereas repeated revision and final-stage review were associated with higher uptake.

Self-reported Confidence and Metacognitive Awareness of Information Uptake

In this section, we examine whether participants’ self-reported confidence reflected their information uptake and whether interaction traces could reveal failures that confidence did not. Participants rated their confidence on a five-point scale after each recognition question. Here, we defined ratings of 4-5 as high confidence and an incorrect high-confidence response as a high-confidence miss. Confidence analyses used all 1187 valid units from 62 participants.

We found that self-reported confidence strongly reflected information uptake. Recognition accuracy increased monotonically from 10.2% at confidence 1 to 92.0% at confidence 5 (see fig:11A). The same pattern held within participants: among the 58 participants with both correct and incorrect units, mean confidence was 4.22 for correct units versus 2.75 for incorrect units, with 50 participants showing higher confidence when correct ($p < 0.001$; see fig:11B). Thus, participants were generally sensitive to whether they had successfully taken up an information unit.

However, self-reported confidence did not capture all uptake failures. We observed 102 high-confidence misses, accounting for 12.3% of high-confidence units and 33.3% of all incorrect units. Moreover, 36 of 62 participants (58.1%) experienced at least one high-confidence miss.

We further examined whether interaction traces could distinguish these failures from high-confidence correct units. Across the 40 workflow-informed interaction features (Section 4.3.3), five showed significant differences in participant-level paired comparisons ($p < 0.05$), all indicating weaker engagement for high-confidence misses than for high-confidence correct units (see fig:11C). Specifically, high-confidence misses showed fewer source checks before Draft revision ($M = 0.57$ vs. $M = 0.99$), a lower minimum interval from source checking to Draft revision ($M = 2.25\ \mathrm{s}$ vs. $M = 9.98\ \mathrm{s}$), a lower likelihood of revisiting the unit in the Draft ($M = 43.3\%$ vs. $M = 56.9\%$), fewer distinct occurrences of the unit visited across Chat and Draft ($M = 1.66$ vs. $M = 2.05$), and fewer unit-linked engagement events (e.g., visits, copying, pasting, selection, editing operations) across the Chat and Draft surfaces ($M = 12.92$ vs. $M = 15.78$). Interaction traces may therefore provide complementary evidence of uptake risk when self-reported confidence alone is misleading.

Discussion

In this section, we discuss the broader implications of our findings for understanding and modeling information uptake in human-LLM co-generation. First, we discuss information uptake as an unfolding interaction process, highlighting sustained active engagement as particularly informative for future HCI user modeling. Second, we discuss how performance-based uptake assessment can be extended to open-ended artifacts. Third, we discuss how low-cost interaction traces can support user-state modeling in dynamic everyday workflows. Fourth, we consider practical applications of our work for motivating the design of uptake-aware human-LLM co-generation systems. Finally, we discuss the limitations of this work and directions for future research.

Information Uptake Unfolds Across the Interaction Process

Across our modeling and behavioral analyses, evidence of information uptake was distributed across the evolving interaction history of an information unit rather than concentrated in a single event, interface, or moment. In our modeling results, preserving multiple semantic occurrences and layout states improved prediction, while combining Chat and Draft traces outperformed either surface alone (Section 5.3). Our behavioral analyses showed a similar pattern: exposure, content adoption, and self-reported confidence each provided incomplete evidence of uptake. Together, these results suggest that information uptake is an unfolding process reflected in how users continue to engage with the same information over time.

This process-level view can inform how HCI systems model user states from interaction behavior. Prior work often summarizes behavior into fixed representations, such as session-level interaction features for predicting knowledge gain [?], page-level statistics for modeling engagement [25], or task-instance signals for modeling AI reliance [60]. Our findings suggest that these signals may become more informative when interpreted in the context of the evolving interaction trajectory of the same information. For example, the same acceptance event may reflect different levels of processing depending on whether the user later revisits that information. Incorporating such process-level evidence could enrich existing models of knowledge gain [68, 75], AI reliance [73, 79], and user engagement [2, 57].

Our behavioral analyses further identify sustained active engagement as a particularly informative family of traces for modeling uptake. Prior work has linked individual behaviors and broader interaction patterns to user outcomes: frequent copy-paste and repeated LLM reference are associated with user reliance [58], while modifying rather than directly accepting AI-generated text has been linked to stronger writing outcomes [90]. Related work also highlights verification and response integration as important forms of critical engagement with AI-generated information [50], and shows that prompting users to reflect can reduce overreliance in AI-assisted decision making [54]. Our findings show that uptake depends not only on whether users encounter or adopt information, but also on how they continue to engage with that same information afterward. Repeated inspection, revision, Chat–Draft coordination, source checking, and final-stage review all reflect continued engagement with the same information over time. Thus, the same copy-paste or acceptance event may carry different evidence depending on whether the user later revisits, revises, or checks that content. Future user models should therefore treat sustained active engagement with the same information as an important behavioral signal, rather than solely relying on whether an action occurred or how often it occurred.

Extending Performance-Based Uptake Assessment to Open-Ended Artifacts

In our study, we adopt a performance-based measure of information uptake following prior work [13, 19, 22, 71]. For each atomic information unit, uptake is labeled by whether the user correctly answers the corresponding post-task recognition question. This provides a directly scored behavioral outcome that complements subjective judgments, which can be affected by individual calibration and remain misleading when users are confident but incorrect [10, 91]. Existing studies often organize measurement around predefined structural units, such as documents [28], paragraphs [78], individual HTML elements [26], or predefined content blocks [98]. However, this creates a gap between lab-based measurement and open-ended real-world tasks, where the final content and assessment units emerge dynamically through interaction. Our approach instead derives atomic assessment targets from each final artifact, preserving the open-ended nature of the task while retaining structured, performance-based assessment. This gives our findings greater real-world relevance and makes them more applicable to actual user workflows.

More broadly, this approach can help researchers bring performance-based assessment into more realistic HCI workflows, such as writing [14], information seeking [17, 64], and planning [2]. By deriving assessment targets from what users actually produce during the task, it enables more personalized and diagnostic evaluation of human-AI systems. Prior HCI work has personalized evaluation or support based on user-level proficiency [77], performance on predefined tasks [88], or predicted user states [57]. These approaches adapt scores, feedback, or interventions within a predefined assessment space. Our approach instead makes the assessment targets themselves user-specific by deriving them from the content each user actually produces. This enables content-level diagnosis of which specific information a user has taken up in an open-ended workflow.

In this work, we use immediate recognition as the performance probe, but the same strategy could extend to other uptake-related outcomes. Delayed recognition or recall could assess whether information remains accessible beyond the immediate task [94]; comprehension assessments could test whether users understand its meaning [42]; and transfer assessments could examine whether users can apply it in new contexts or tasks [84]. Extending the same artifact-adaptive strategy across these outcomes could provide a broader view of how deeply and durably users take up information.

From Specialized Sensing to Low-Cost Interaction Traces in Everyday Workflows

TraceMind suggests a practical path toward uptake-aware systems by relying on low-cost interaction traces. Mouse, keyboard, and interface-level traces are already produced during ordinary browser and editor use, without requiring dedicated sensing hardware, continuous camera access, or repeated self-report [11, 34]. This makes them promising signals for integrating user-state modeling into everyday co-generation workflows.

Our results suggest that interaction events in dynamic co-generation interfaces should be anchored to both the semantic identity of the information and its screen layout. Prior web interaction work typically ties behavior to page elements or spatial regions: viewport exposure has been used to model attention over individual page elements [26], mouse activity has been aligned with content blocks [8], and cursor-based models have incorporated layout-specific features for heterogeneous SERP layouts [87]. In these settings, the page element often serves as both the content identifier and the spatial anchor. This coupling breaks in co-generation: the same information may be paraphrased, move from Chat to Draft, and later reflow into a different geometry. TraceMind therefore performs two distinct alignment steps: semantic alignment identifies spans that express the same information across Chat and Draft, while layout alignment recovers where each matched span was rendered over time. Our ablations show that both contribute to prediction. This suggests that interaction modeling in editable, generative interfaces should track semantic identity and rendered geometry separately rather than assume that a stable interface element can represent both.

In addition, our soft-boundary analysis extends the idea of soft bounding regions from prior work to low-cost interaction-trace modeling. Eye-tracking studies commonly expand dynamic areas of interest beyond their exact bounding boxes to avoid missing relevant gaze samples [18], while prior analyses show a trade-off between capturing relevant activity and introducing false associations as AOI margins increase [70]. We observe a similar pattern for low-cost pointer traces: restricting pointer trajectories to the exact target bounding box substantially weakened prediction, whereas retaining a moderate amount of nearby activity provided useful additional evidence. Expanding the region further, however, offered little additional benefit. These results suggest that low-cost interaction modeling should use bounded, target-relative spatial context rather than treat pointer activity as a binary inside-or-outside event.

Finally, information uptake is only one user state that can be modeled from low-cost interaction traces. Prior work has used mouse and interface behavior to infer attention [4], engagement [25], and knowledge gain [?]. Our results suggest that combining semantic and layout alignment could extend such modeling to more dynamic tasks and interface settings, helping future HCI systems unobtrusively infer users’ evolving states during everyday interaction.

Toward Uptake-Aware Human-LLM Co-Generation Systems

Our findings motivate uptake-aware human–LLM co-generation systems, particularly in consequential settings where users must share, submit, or act on AI-assisted content. Beyond existing support for generating and refining content [12, 41, 59], uptake-aware systems could surface information with weak evidence of uptake for targeted review, making human-LLM collaboration more transparent while preserving user agency and meaningful oversight.

This uptake-aware layer could complement existing HCI support for AI-assisted writing, which has largely focused on suggestions, writing scaffolds, and co-writing workflows [16, 52, 66]. In accuracy-sensitive settings, such as scientific writing [66] and contract drafting [95], uptake-aware support could prioritize claims with weak evidence of uptake for verification, helping users focus review effort where accuracy matters most. In public-facing communication, such as journalism [69] and online posting [37], it could surface AI-assisted content with weak evidence of uptake before publication, helping authors remain aware of and in control of what they communicate. Across these settings, uptake awareness complements support for content production by making users’ engagement with the resulting information itself an explicit design concern.

This uptake-aware layer could also extend beyond content production to settings where users rely on LLM-provided information. In LLM-assisted learning, where monitoring one’s own learning is critical [49], uptake-aware support could infer which information learners have not fully taken up, enabling more targeted and personalized learning support. In AI-assisted decision making, where overreliance on AI remains a concern [54, 55], uptake-aware signals could infer when users rely on decision-relevant information they have not fully processed, supporting more critical evaluation of AI advice.

At the same time, uptake-aware support should remain soft, inspectable, and privacy-conscious. Because interaction traces provide probabilistic rather than definitive evidence of uptake, predictions should flag content for possible review rather than judge what users know or understand. Systems could consider information importance and interruption cost when deciding whether to intervene [43, 63], and use occasional user feedback to calibrate such support [82]. Although TraceMind relies only on low-cost interaction traces rather than cameras or other dedicated sensing, these traces may still reveal sensitive content and behavioral patterns. Practical systems should therefore minimize data collection, favor local or on-device processing when feasible [39, 72], and give users control over what data are recorded and retained [3].

Limitations and Future Work

First, following prior work, we operationalize information uptake as users’ immediate recognition of atomic information units in the final draft. This captures recognition-level uptake, but not comprehension, application, or longer-term retention. Future work should examine whether the same interaction traces generalize to these outcomes. Second, post-task recognition questions were generated and screened through an automated LLM-based pipeline without independent human validation. While this design enabled assessment to adapt to each open-ended artifact, automatically generated items may not achieve psychometric quality and item discrimination comparable to expert-written items [30, 47]. Future studies should further combine blinded expert review with empirical item analysis. Third, our participants were predominantly experienced LLM users in China, and our study covered three representative everyday writing tasks. Future work should evaluate TraceMind across more diverse populations and tasks. Finally, we prioritized low-cost interaction traces over specialized sensing hardware. In our study, participants used laptops with external mice to ensure consistent capture. Future work should test whether TraceMind transfers to trackpads, touchscreens, and other input modalities.

Conclusion

Information uptake is a complex, unfolding process that deserves greater attention to how human-LLM systems are evaluated and designed. By extending performance-based uptake assessment to open-ended artifacts and aligning low-cost interaction traces with atomic information units, our work provides a concrete approach for assessing and modeling uptake during co-generation. We encourage future HCI research to move beyond whether AI-generated content is adopted and toward better ways of assessing and supporting what users actually take up, enabling uptake-aware human-LLM co-generation systems grounded in low-cost interaction traces.

References

A DATA COLLECTION STUDY DETAILS

This section provides additional details about the data collection study, including participant demographics and the complete participant-facing instructions for the three co-generation tasks.

Participant Demographics

We provide the participants’ demographics and LLM usage background (see Table 5).

Participant-Facing Task Instructions

We provide the complete participant-facing instructions for the three co-generation tasks. The instructions were originally presented in Chinese and are reproduced here in English. To preserve anonymity, we replaced potentially identifying proper nouns, such as universities, locations, venues, and speaker names, with consistent placeholders while preserving the original task scenarios.

Club Team-Building Announcement.

You are responsible for organizing your club’s annual team-building activity, which aims to promote communication and collaboration among members. The following arrangements have been confirmed:

• During the day, the group will travel by coach from University A to Historic Town H and visit Temple T, the South Gate, and the historic district.

• In the evening, the group will dine at Restaurant R and try the town’s local “Eight Bowls” cuisine before returning together by coach.

Please use an LLM to help you draft a team-building announcement for all club members. The announcement should introduce the purpose of the activity, the complete itinerary, the attractions, and the restaurant. It should also include practical information that participants may need, such as transportation arrangements, attire suggestions, an introduction to the local cuisine, activity preparation, and other relevant precautions.

Submit a complete team-building announcement that you believe could be sent directly to club members. Treat the final draft as a formal announcement ready for public release and prepare it carefully.

Roommate Recruitment Post.

You will attend a two-month summer program at University B and stay in a shared apartment. The following arrangements have been confirmed:

• Apartment A is located in District D.

• The unit type is $D2-4 \times4$. You and two other students in a joint program between University A and University B have confirmed your stay, and you need to recruit one additional roommate.

Please use an LLM to help you draft a recruitment post for potential roommates. You may introduce the apartment, bedrooms, amenities, surrounding neighborhood, transportation, and move-in arrangements to help potential roommates understand the living situation. You should also describe your basic expectations for shared living, such as how expenses will be divided, cleaning, visitors, daily schedules, or other house rules.

Submit a complete roommate recruitment post that you believe could be published publicly. Treat the final draft as a formal post ready for public release and prepare it carefully.

Lecture Announcement.

The university will hold an “AI Lecture Series” event at 2:00 p.m. the following Friday. Three academicians from University A—Scholar P1, Scholar P2, and Scholar P3—will discuss the current state and future development of artificial intelligence. Each scholar will give an academic talk, followed by a Q&A session.

Please use an LLM to help you draft an event announcement for all students at the university. Briefly introduce each scholar, including, for example, their research areas and representative work. The announcement should also include information that attendees may need, such as the event theme, the value of attending, suggested preparation, and topics they may wish to follow or ask about.

Submit a complete announcement that you believe could be sent directly to all students at the university. Treat the final draft as a formal announcement ready for public release and prepare it carefully.

B TRACEMIND FRAMEWORK DETAILS

Workflow-informed Interaction Features

We provide the complete set of workflow-informed interaction features used in TraceMind. Table 6 lists all 40 features, organized by the eight behavior types.

PID	Gender	Age	Education	LLM Familiarity	LLM Use Frequency (past month)	LLM Writing Frequency	LLM Reliance Degree	LLM Use Experience
P1	Male	21	Undergraduate	FA	Multiple times daily	Always	LF	2-3 years
P2	Male	20	Undergraduate	FA	Multiple times daily	Always	LF	1-2 years
P3	Female	22	Undergraduate	VF	Multiple times daily	Always	LD	2-3 years
P4	Female	19	Undergraduate	VF	Multiple times daily	Always	LD	2-3 years
P5	Female	19	Undergraduate	FA	Multiple times daily	Always	LD	1-2 years
P6	Male	23	Doctoral	VF	Multiple times daily	Always	CL	2-3 years
P7	Female	23	Master’s	FA	A few times per week	Sometimes	CL	2-3 years
P8	Male	26	Master’s	FA	Multiple times daily	Always	LF	1-2 years
P9	Female	21	Undergraduate	FA	A few times per week	Often	CL	2-3 years
P10	Male	20	Undergraduate	FA	A few times per week	Often	LF	2-3 years
P11	Female	21	Undergraduate	FA	Almost daily	Often	CL	1-2 years
P12	Female	18	Undergraduate	FA	Almost daily	Often	LF	2-3 years
P13	Male	22	Master’s	FA	Multiple times daily	Always	LD	1-2 years
P14	Male	19	Undergraduate	FA	A few times per week	Often	LD	7-12 months
P15	Male	21	Undergraduate	FA	Almost daily	Often	CL	1-2 years
P16	Male	22	Undergraduate	FA	Almost daily	Sometimes	CL	2-3 years
P17	Female	19	Undergraduate	FA	Almost daily	Often	CL	1-2 years
P18	Female	20	Undergraduate	FA	A few times per month	Always	LD	7-12 months
P19	Female	20	Undergraduate	FA	Almost daily	Often	CL	1-2 years
P20	Male	22	Master’s	VF	Multiple times daily	Always	CL	More than 3 years
P21	Male	24	Master’s	OU	Almost daily	Often	CL	1-2 years
P22	Female	23	Master’s	VF	Multiple times daily	Always	LD	2-3 years
P23	Male	22	Undergraduate	OU	A few times per month	Often	LF	2-3 years
P24	Female	22	Master’s	VF	Multiple times daily	Always	LD	More than 3 years
P25	Female	30	Undergraduate	OU	Almost daily	Sometimes	LF	1-2 years
P26	Male	21	Undergraduate	FA	Almost daily	Always	CL	2-3 years
P27	Male	23	Master’s	FA	Almost daily	Sometimes	CL	1-2 years
P28	Male	21	Undergraduate	FA	Almost daily	Often	CL	1-2 years
P29	Female	21	Undergraduate	FA	Multiple times daily	Always	LD	1-2 years
P30	Female	23	Undergraduate	FA	A few times per week	Often	CL	1-2 years
P31	Female	23	Master’s	FA	Multiple times daily	Always	LD	2-3 years
P32	Female	23	Master’s	FA	Almost daily	Sometimes	LF	2-3 years
P33	Male	21	Undergraduate	FA	Almost daily	Always	LD	More than 3 years
P34	Female	24	Master’s	FA	Multiple times daily	Always	LF	More than 3 years
P35	Male	20	Undergraduate	FA	A few times per week	Sometimes	LF	1-2 years
P36	Female	23	Undergraduate	FA	A few times per week	Always	LD	2-3 years
P37	Male	20	Undergraduate	FA	Almost daily	Sometimes	CL	1-2 years
P38	Female	23	Undergraduate	VF	Almost daily	Always	LF	2-3 years
P39	Male	21	Undergraduate	VF	Almost daily	Often	LF	More than 3 years
P40	Male	23	Undergraduate	VF	Multiple times daily	Often	CL	2-3 years
P41	Female	20	Undergraduate	FA	Almost daily	Often	LD	1-2 years
P42	Male	20	Undergraduate	FA	A few times per week	Sometimes	MS	2-3 years
P43	Female	20	Undergraduate	FA	Almost daily	Often	LF	2-3 years
P44	Male	20	Undergraduate	FA	Multiple times daily	Always	LD	2-3 years
P45	Female	22	Undergraduate	FA	Multiple times daily	Often	LF	2-3 years
P46	Male	23	Undergraduate	FA	Multiple times daily	Often	LF	1-2 years
P47	Male	20	Undergraduate	FA	A few times per week	Often	CL	Less than 6 months
P48	Male	21	Undergraduate	FA	Multiple times daily	Often	LF	1-2 years
P49	Female	21	Undergraduate	FA	Multiple times daily	Always	LD	2-3 years
P50	Female	19	Undergraduate	FA	Almost daily	Often	CL	7-12 months
P51	Male	20	Undergraduate	FA	A few times per week	Sometimes	LF	7-12 months
P52	Male	21	Undergraduate	FA	Multiple times daily	Often	CL	1-2 years
P53	Female	20	Undergraduate	FA	Multiple times daily	Often	CL	2-3 years
P54	Male	22	Master’s	FA	Multiple times daily	Always	LD	1-2 years
P55	Male	21	Undergraduate	FA	Almost daily	Always	LD	1-2 years
P56	Female	20	Undergraduate	FA	Multiple times daily	Often	CL	1-2 years
P57	Female	22	Undergraduate	FA	Multiple times daily	Often	CL	1-2 years
P58	Male	26	Master’s	FA	A few times per week	Often	CL	1-2 years
P59	Female	21	Undergraduate	FA	Almost daily	Often	LF	1-2 years
P60	Female	23	Undergraduate	FA	Multiple times daily	Often	LD	2-3 years
P61	Male	20	Undergraduate	FA	Almost daily	Often	LF	7-12 months
P62	Male	21	Undergraduate	FA	Multiple times daily	Often	LF	1-2 years

TRACEMIND FRAMEWORK DETAILS