betaFeedback

WorksComputer Science

Scaffolding GenAI Use in Ethical Essay Writing: A Three-Year Experience Report in Introductory Computing

Abstract

As society, ethics, and professional practice become increasingly integrated across the computer science curriculum, technical courses need practical ways to support ethical reasoning. Ethical essay assignments offer one approach, but little is known about how students use GenAI when writing such essays. This experience report examines GenAI use across three years of ethical essay assignments in a large first-year computer systems course. Across six post-essay surveys from 2024 to 2026, we analyzed 1,894 responses, linked survey data with essay grades in 2025 and 2026, and introduced a scaffolded responsible-use workflow in 2026. Under voluntary-use policies, AI adoption remained steady at around two-thirds of students, rising above 90% when use was scaffolded. Students reported mainly using AI for help with grammar and style, while tool use diversified beyond ChatGPT in 2026. Grade-linked analyses showed that AI use did not independently predict essay quality, while marker evaluations indicated high levels of critical engagement when use was scaffolded. Overall, GenAI's pedagogical value appears to depend on how assessment design guides students to evaluate and critically engage with AI-generated suggestions.

CCS Concepts

• Social and professional topics → Computing education.

Introduction and Background

Generative AI (GenAI) tools have rapidly become part of the higher education student experience since the public release of ChatGPT in late 2022 [13]. In computing education, students use these tools not only for programming, but also for brainstorming, writing support, and language refinement, with studies reporting both enthusiasm and skepticism [6, 9, 14].

This shift coincides with the growing importance of society, ethics, and professional practice in computing education curricula. The CS2023 guidelines highlight the rising prominence of societal and ethical concerns and recommend integrating the Society, Ethics, and the Profession knowledge area across the curriculum from Year 1, including within technical courses [10]. Recent work echoes this interest while noting persistent barriers such as time constraints, limited expertise, and competing priorities [12, 16, 17]. Ethical essays are a common assignment form because they require students to apply normative theories, identify stakeholders, and justify positions on contentious issues [11], and GenAI heightens their relevance by introducing concerns such as algorithmic bias, misinformation, and autonomous decision-making [1]. Short ethical essays in an introductory computing systems course therefore offer a useful setting for studying GenAI-supported writing.

However, much of the existing empirical evidence on student GenAI use comes from cross-sectional studies of single cohorts at a single point in time. While these snapshot studies provide valuable early insights, they cannot show how student practices change as tools, policies, and pedagogies mature. Several foundational studies call for longitudinal research on changes in student behaviour over time [3, 5, 8], a gap also noted in systematic reviews of GenAI in higher education [7]. Our study is not a longitudinal panel; however, it follows three successive cohorts of the same course (a repeated cross-sectional design), showing how practices changed at the course level rather than for individual students.

Equally important is how students engage with AI output. Yang et al. [18] show that actively modifying AI-generated text yields higher-quality writing, whereas passively accepting suggestions does not. But this causal evidence comes from controlled settings with crowd-sourced participants, leaving open whether the same holds in authentic coursework, where students write under time constraints, on assigned topics, with varied tools.

This experience report addresses these gaps through a three-year study (2024–2026) in a large first-year introductory computing course. Across six essays, we analyse:

N=448⏟2024+694⏟2025+752⏟2026=1,894 student responsesN = \underbrace{448}_{2024} + \underbrace{694}_{2025} + \underbrace{752}_{2026} = 1{,}894\ \text{student responses}

examining how students use GenAI for ethical essay writing, a format widely used in CS courses [4, 11]. Ethical essays are a useful context because GenAI can support writing form, but also invite uncritical content and unsupported arguments. We therefore examine not only whether students use AI, but how they engage with it—whether they modify, reject, or passively accept suggestions. Scaffolding and AI disclosure are not new in themselves; our contribution is to track GenAI use in computing-ethics writing across three cohorts of an authentic course, link it to grades, and evaluate a draft-first scaffold through self-reports and marker coding. We report (1) adoption patterns under voluntary (2024–2025) and scaffolded (2026) policies, (2) a reusable responsible-use scaffold, and (3) practical lessons for educators.

Motivation and Study Arc

The motivation for this study arose from an unexpected observation in our introductory computing systems course. In Semester 1, 2024, we expressly allowed students (≈ 570) to use GenAI tools for two ethics essays, with mandatory disclosure. Contrary to expectations, many students reported not using AI at all. This gap between permission and uptake prompted our central motivating question:

Motivating Question: How do students actually engage with GenAI when given the freedom to use it, and how does this engagement change over time?

Our investigation unfolded in three phases (cf. Table 1).

AttributePhase 1 (2024)Phase 2 (2025)Phase 3 (2026)
Study:ExplorationMeasurementRedesign
#Enrolled:570409425
#Responses:448 (236+212)694 (360+334)752 (388+364)
Response Rate:85%85%90%
Grade link:No (0)Yes (316)Yes (342)
#Markers:6109
AI Policy:OptionalOptionalScaffolded
Essay 1 Topic:Digital addictionDigital addictionAI hiring ethics
Essay 2 Topic:Job displacementJob displacementSoc.Media/health

Table 1. Study overview across three phases.

  • Phase 1 (2024): Exploration: explored voluntary adoption, including tool choices, and purposes of use.

  • Phase 2 (2025): Measurement: retained the same survey instrument but linked responses to tutor-assessed essay grades.

  • Phase 3 (2026): Scaffolded Redesign: introduced a scaffolded responsible-use workflow and marker-facing engagement codes.

This progression, from exploration to measurement to pedagogical redesign, follows a design-based research arc [2] in which each phase was informed by the findings of the previous one.

Evaluation approach

We evaluated the way that GenAI use was adopted by students for essay writing across three years of course offerings. We were interested in understanding what changed over time, whether AI use explained essay performance, and what scaffolded engagement with AI looked like in student submissions.

We organize our evaluation around three guiding questions:

  • Q1 (Evolution): How have GenAI adoption patterns, tool preferences, purposes of use, and selected student perceptions evolved across three years of the same course?

  • Q2 (Essay Quality): Does GenAI use predict essay quality once prior writing ability is controlled, and do adoption trajectories predict score changes between essays?

  • Q3 (Critical Engagement): How do students engage with AI suggestions under scaffolded conditions, and what evidence of critical engagement is visible in their submissions?

Context and Participants

The study was conducted in Introduction to Computer Systems, a mandatory first-year course in a CS degree covering standard systems topics such as binary representation, computer architecture, programming languages, operating systems, and networking. The course runs twice annually, enrolling approximately 500–600 students in the first semester and 200–300 in the second.

Participants were first-year undergraduate students enrolled in the course across three consecutive offerings. Table 1 summarises participation across the three phases.

Tasks and Assessment

Each semester, the course includes two short essay assignments (400–500 words) on ethical issues in computing (Table 1). They are the only assessment for the course’s ethics learning outcome, “Discuss ethical issues arising from the use of computers in society”, and are worth 8% of the final grade. The essays run alongside the technical content as a separate strand rather than being tied to particular lectures. Because most students are school leavers, the essays are also their first academic writing in the CS degree. Students are asked to interpret a given prompt, construct a reasoned argument, identify stakeholders, and support their positions with credible references.

Teaching assistants marked the essays (Table 1) following written marking instructions and a calibration discussion held before marking began. The 2024–2025 rubric awarded 7 points, one each for clarity, organisation, evidence, ethical reasoning, referencing, AI disclosure, and length. The 2026 rubric awarded 16 points across three levels, adding reflection (4) and weighting references (4) more heavily; the draft and disclosure form were required but unmarked. A strong essay was one that explained the topic in the student’s own words, analysed stakeholders, and cited evidence; the 2026 rubric also required a clear position, two or more stakeholders, and a counterargument. Full rubrics are available at [15].

The 2026 Scaffolded Redesign

The 2026 offering introduced a scaffolded workflow for both essay assignments, informed by findings from Phases 1 and 2. Each element of the workflow targeted a specific risk identified in the earlier phases (Table 2). The workflow asked students to: (1) write a complete draft independently, with a checklist at its top; (2) ask an AI tool of their choice, using their own prompt, to refine (not rewrite) the draft; (3) accept suggestions that improved clarity and reject any that changed their argument; and (4) make any final edits, submit, and complete the AI disclosure form. Non-users still submitted a draft. The scaffolded workflow is publicly available [15].

Risk identifiedScaffolded requirement
Students may outsource content to AIMandatory independent draft before any AI use, submitted with and linked to the final essay.
Students may use AI before forming an ethical argumentDraft checklist requiring students to identify at least two stakeholders, include a counterargument, and flag claims needing evidence.
Students may not verify AI suggestionsAt least two credible references, with every claim needing evidence cited, as AI output may be inaccurate.
Students may not reflect on AI interactionStructured reflection (100–200 words) on AI use, differences between draft and final essay, and potential improvements.
Students may not disclose AI useSeparate AI disclosure form required; a missing draft or form resulted in zero marks.

Table 2. Workflow design rationale and scaffold details.

To support the workflow, students submitted four parts—(P1) a final essay (50% mark), (P2) a pre-AI draft written in their own words (not marked but mandatory), (P3) a structured reflection (25%), and (P4) an IEEE-formatted reference list (25%)—along with a separately submitted (P5) AI disclosure form (Table 2). The draft, reflection, and final essay were submitted together; hence, markers could compare all three side by side. Two scaffold elements structured how students prepared and reflected on their writing:

  • A four-item checklist at the top of the draft, in which students confirmed they had (i) written the draft in their own words, (ii) identified at least two stakeholders, (iii) included at least one counterargument, and (iv) flagged claims needing supporting evidence.

  • A fixed four-prompt reflection covering (i) their initial reading of the prompt, (ii) whether and how they used AI, and what was most and least useful, (iii) the main difference between their draft and final essay, and (iv) what they would do differently next time.

The combined submission (draft, reflection, and final essay) enabled three additional categorical codes that markers recorded:

  • AI tone (Positive, Neutral, Negative, or Did not use AI) records the marker’s impression of how AI use affected the submission.

  • Critical engagement (Yes, No, or Did not use AI) captures whether the student showed evidence of critically evaluating AI output, e.g., by describing what they rejected or modified, or by leaving visible traces of independent judgement in the final essay.

  • Genuine insight (Yes or No) captures whether the draft-to-final transition shows substantive development of the student’s reasoning rather than surface-level edits.

Codes were applied to both essays (N=383N = 383 for Essay 1, N=358N = 358 for Essay 2 after linking with surveys). The codes were covered by the same written instructions and calibration discussion.

Data Collection and Analysis

After each essay, students completed a post-essay survey via Microsoft Forms. Across all phases it captured AI use, tool choice, purposes of use, and three perception measures: trust in AI output,

intention to reuse AI, and perceived improvement in essay quality. Responses were linked with essay grades in Phases 2 and 3, and in Phase 3 the survey was expanded to capture responsible-use behaviours such as reference verification and rejection or modification of AI suggestions. All analyses are structured around the three evaluation questions in Section 2:

For Q1, we report descriptive statistics for adoption, tool, and purposes of use, and use proportional Z-tests to identify statistically significant cross-year shifts. All cross-year comparisons (except adoption rates) are based on post-Essay 1 responses, because it is the survey administered most consistently across all three offerings.

For Q2, we use Phase 2 and Phase 3 grade-linked data. An analysis of covariance (ANCOVA) models Essay 2 rubric scores as the dependent variable, AI use in Essay 2 as the predictor, and Essay 1 score as a baseline covariate. We also examine score changes (ΔEssay = Essay 2 − Essay 1) by adoption trajectory:

  • Always: used AI for both Essay 1 as well as Essay 2;

  • Never: did not use AI for either Essay 1 or Essay 2;

  • Adopt: did not use AI for Essay 1, but adopted it for Essay 2;

  • Abandon: used AI for Essay 1, but abandoned it for Essay 2.

For Q3, we focus on Phase 3 evidence of critical engagement under scaffolded use. We report student self-reports of rejection and modification as supporting evidence, and foreground the marker codes for AI tone, critical engagement, and genuine insight applied to Essay 1 and Essay 2 submissions. All statistical tests are two-tailed and use a 95% confidence level.

Findings

AI Adoption, Tools, and Purposes of Use (Q1)

Q1 examined how students’ AI adoption, tool preferences, and purposes of use evolved across three course offerings. All cross-year comparisons in this section use deduplicated valid survey responses. Because the 2026 offering differed from 2024 and 2025 in that students were expected to engage with AI under a more scaffolded design, we interpret the 2026 comparisons with caution.

Adoption. We define adoption as answering Yes to “Did you use any generative AI tool to assist in improving your essay?”, an item asked identically in all three years. It covers any AI use, whatever its purpose, and in 2026 includes the workflow’s refinement step; purposes of use are reported separately below. Under voluntary use, adoption did not change significantly: 59% to 64% for Essay 1 and 69% to 68% for Essay 2 (2024 to 2025). In 2026 it reached 93% for both essays (both p<.001p < .001 vs. 2025). Because 2026 introduced an expected-use, structured workflow, this is not organic growth; it suggests that assessment design and policy framing strongly shape whether students use AI.

RESULT 1A (Policy, not time, drives adoption): Under voluntary policy, roughly 2/3 of students chose to use AI in both 2024 and 2025, with no significant year-on-year change. Under the 2026 expected-use policy, adoption rose sharply to over 9/10 students for both essays.

Tool choice. ChatGPT dominated in all three years but its share fell sharply in 2026 (Figure 1). Across 2024 and 2025 it accounted for roughly 90% of primary tool use in both essays (E1: 95% and 91%; E2: 92% and 88%; year-on-year changes non-significant), then dropped to 64% (Essay 1) and 59% (Essay 2) in 2026, a significant decline in both. The shift reflects diversification rather than reduced AI use: Claude and Gemini, near-absent earlier, each captured roughly one-tenth of users in 2026, and Grammarly rose to around 7%. Copilot held steady at about 2% throughout.

Bar chart showing primary AI tool for Essay 1 across 2024, 2025, and 2026, with ChatGPT, Claude, Gemini, Grammarly, Copilot, and Other

Figure 1. Primary AI tool for Essay 1, across the three phases.

Bar chart showing self-reported task use for Essay 1 across 2024, 2025, and 2026, with Style / Grammar, Info/Research, Brainstorming, and Structure/Outline

Figure 2. Self-reported task use for Essay 1

Result 1b (ChatGPT’s near-monopoly broken): ChatGPT’s share dropped from around 90% in 2024–2025 to around 60% in 2026 across both essays. Tool diversification, not reduced AI use, drove the shift: collectively, Claude, Gemini, and Grammarly captured roughly 1/3 of users in 2026.

Purpose of use. Across all three phases, students used AI primarily for writing support rather than content generation (Figure 2). 65% of AI users reported using AI for style, grammar, and form in 2024 and 61% in 2025, rising to 91% in 2026—consistent with the 2026 redesign, though the changed instructional context means this is not a direct year-on-year shift. Among categories available in both 2025 and 2026, brainstorming fell sharply from 47% to 15%, while structure and outlining held steady (41% to 35%). These patterns suggest students increasingly used AI for revision and presentation rather than ideation or wholesale content generation.

Result 1c (Students use AI primarily for form): Across all three years, students used AI primarily for grammar and form (Essay 1: 65% → 61% → 91%). Brainstorming fell from 47% in 2025 to 15% in 2026, while structure and outlining held relatively steady.

Figure 3. Changes in student perceptions among AI users across three years. Significance markers indicate year-on-year comparisons: * p<.05p < .05, *** p<.001p < .001, n.s. = not significant

Student perceptions. Student perceptions did not become uniformly more positive over time (Figure 3). Trust in AI output rose from 7.9% of AI users in 2024 to 22% in 2025 (p<.001p < .001), then held at 22% in 2026, suggesting that familiarity did not lead to uncritical acceptance. Reuse intention and perceived quality instead showed a dip-and-recovery pattern: intended reuse fell from 41% (2024) to 30% (2025) before rising to 49% (2026), while perceived quality moved from 64% to 59% to 77%. This may reflect early enthusiasm giving way to more measured views in 2025, then renewed confidence under the scaffolded 2026 workflow.

Taken together, Q1 shows that AI use did not simply increase with time. Under voluntary-use policies, adoption stabilised at roughly two-thirds of students; under expected and scaffolded use, adoption rose above 90%. At the same time, the tool ecology diversified beyond ChatGPT, and students’ reported purposes remained concentrated on writing support rather than content generation.

AI Use and Essay Quality (Q2)

Q2 examined whether AI use predicted essay quality after controlling for prior writing performance, and whether adoption trajectories between Essay 1 and Essay 2 explained changes in essay scores. Phase 2 (unscaffolded) and Phase 3 (scaffolded) yielded consistent results: AI use did not independently predict essay quality once prior performance was held constant, and adoption trajectories did not significantly explain score changes.

ANCOVA. We modelled Essay 2 score as a function of Essay 1 score (covariate) and AI use in Essay 2 (a focal predictor) in each phase (Table 3). Prior writing ability was a strong predictor in both phases (β=0.245\beta= 0.245 and 0.290, both p<.0001p < .0001), but AI use had no independent effect (Phase 2: β=0.051\beta= 0.051, p=.60p = .60; Phase 3: β=0.350\beta= 0.350, p=.30p = .30, 95% CI [−0.32-0.32, +1.02]). In Phase 3, the raw comparison agreed: AI users averaged 14.83 on the 16-point rubric versus 14.50 for non-users, non-significant under both Welch tt-test (p=.36p = .36) and Mann–Whitney (p=.17p = .17). Neither phase produced an AI-use effect once prior writing ability was controlled.

PhasePredictorCoef.SEttpp-value
Phase 2Essay 1 score0.2450.0514.84p<0.0001p < 0.0001
(N=333)(N=333)AI used in Essay 20.0510.0970.53p=0.60p = 0.60
Phase 3Essay 1 score0.2900.0446.58p<0.0001p < 0.0001
(N=343)(N=343)AI used in Essay 20.3500.3391.03p=0.30p = 0.30

Table 3. ANCOVA predicting Essay 2 score from Essay 1 score and AI use in Essay 2. Phase 2 used a 7-point rubric; Phase 3 used a 16-point rubric.

Result 2a (No independent AI effect on essay quality): After controlling for Essay 1 score, AI use in Essay 2 had no significant effect on Essay 2 quality in either phase. Prior writing ability dominated in both phases.

Adoption trajectories. We grouped students into four AI-use trajectories across the two essays: Always (yes → yes), Never (no → no), Adopt (no → yes), and Abandon (yes → no). Score changes across the four groups are shown in Figure 4. Phase 2 effects were

Mean change in essay score by AI-use trajectory for Phase 2 and Phase 3

Figure 4. Mean change in essay score by AI-use trajectory for Phase 2 (2025) and Phase 3 (2026). Bars show the within-student change in essay score, calculated as Δ\DeltaEssay score = Essay 2 - Essay 1, expressed in % points of the essay scale.

@page 4/7 starts_mid=1 ends_mid=1

Bar chart showing primary AI tool for Essay 1 across 2024, 2025, and 2026, with ChatGPT, Claude, Gemini, Grammarly, Copilot, and Other

Figure 1 (PDF p. 4). Primary AI tool for Essay 1, across the three phases.

Bar chart showing self-reported task use for Essay 1 across 2024, 2025, and 2026, with Style / Grammar, Info/Research, Brainstorming, and Structure/Outline

Figure 2 (PDF p. 4). Self-reported task use for Essay 1

to 64% (Essay 1) and 59% (Essay 2) in 2026, a significant decline in both. The shift reflects diversification rather than reduced AI use: Claude and Gemini, near-absent earlier, each captured roughly one-tenth of users in 2026, and Grammarly rose to around 7%. Copilot held steady at about 2% throughout.

Result 1b (ChatGPT’s near-monopoly broken): ChatGPT’s share dropped from around 90% in 2024–2025 to around 60% in 2026 across both essays. Tool diversification, not reduced AI use, drove the shift: collectively, Claude, Gemini, and Grammarly captured roughly 1/3 of users in 2026.

Purpose of use. Across all three phases, students used AI primarily for writing support rather than content generation (Figure 2). 65% of AI users reported using AI for style, grammar, and form in 2024 and 61% in 2025, rising to 91% in 2026—consistent with the 2026 redesign, though the changed instructional context means this is not a direct year-on-year shift. Among categories available in both 2025 and 2026, brainstorming fell sharply from 47% to 15%, while structure and outlining held steady (41% to 35%). These patterns suggest students increasingly used AI for revision and presentation rather than ideation or wholesale content generation.

Result 1c (Students use AI primarily for form): Across all three years, students used AI primarily for grammar and form (Essay 1: 65% → 61% → 91%). Brainstorming fell from 47% in 2025 to 15% in 2026, while structure and outlining held relatively steady.

Figure 3 (PDF p. 4). Changes in student perceptions among AI users across three years. Significance markers indicate year-on-year comparisons: * p<.05p < .05, *** p<.001p < .001, n.s. = not significant

Student perceptions. Student perceptions did not become uniformly more positive over time (Figure 3). Trust in AI output rose from 7.9% of AI users in 2024 to 22% in 2025 (p<.001p < .001), then held at 22% in 2026, suggesting that familiarity did not lead to uncritical acceptance. Reuse intention and perceived quality instead showed a dip-and-recovery pattern: intended reuse fell from 41% (2024) to 30% (2025) before rising to 49% (2026), while perceived quality moved from 64% to 59% to 77%. This may reflect early enthusiasm giving way to more measured views in 2025, then renewed confidence under the scaffolded 2026 workflow.

Taken together, Q1 shows that AI use did not simply increase with time. Under voluntary-use policies, adoption stabilised at roughly two-thirds of students; under expected and scaffolded use, adoption rose above 90%. At the same time, the tool ecology diversified beyond ChatGPT, and students’ reported purposes remained concentrated on writing support rather than content generation.

AI Use and Essay Quality (Q2)

Q2 examined whether AI use predicted essay quality after controlling for prior writing performance, and whether adoption trajectories between Essay 1 and Essay 2 explained changes in essay scores. Phase 2 (unscaffolded) and Phase 3 (scaffolded) yielded consistent results: AI use did not independently predict essay quality once prior performance was held constant, and adoption trajectories did not significantly explain score changes.

ANCOVA. We modelled Essay 2 score as a function of Essay 1 score (covariate) and AI use in Essay 2 (a focal predictor) in each phase (Table 3). Prior writing ability was a strong predictor in both phases (β=0.245\beta= 0.245 and 0.290, both p<.0001p < .0001), but AI use had no independent effect (Phase 2: β=0.051\beta= 0.051, p=.60p = .60; Phase 3: β=0.350\beta= 0.350, p=.30p = .30, 95% CI [−0.32-0.32, +1.02]). In Phase 3, the raw comparison agreed: AI users averaged 14.83 on the 16-point rubric versus 14.50 for non-users, non-significant under both Welch tt-test (p=.36p = .36) and Mann–Whitney (p=.17p = .17). Neither phase produced an AI-use effect once prior writing ability was controlled.

Result 2a (No independent AI effect on essay quality): After controlling for Essay 1 score, AI use in Essay 2 had no significant effect on Essay 2 quality in either phase. Prior writing ability dominated in both phases.

Adoption trajectories. We grouped students into four AI-use trajectories across the two essays: Always (yes → yes), Never (no → no), Adopt (no → yes), and Abandon (yes → no). Score changes across the four groups are shown in Figure 4. Phase 2 effects were uniformly small (−0.23-0.23 to +0.14+0.14 on the 7-point rubric), and the overall effect was not significant (F=1.25F = 1.25, p=.29p = .29). Phase 3 showed directionally larger effects—Adopters +0.88+0.88, Abandoners −0.50-0.50 on the 16-point rubric—but the Adopt (n=12n = 12) and Abandon (n=9n = 9) groups were too small to distinguish from noise, and the overall trajectory effect was again non-significant under one-way ANOVA (F=0.86F = 0.86, p=.46p = .46) and Kruskal–Wallis (p=.51p = .51). Neither phase supports a strong trajectory-based effect.

Result 2b (No trajectory effect in either phase): Score changes did not differ significantly across the four adoption trajectories in either phase. In Phase 3, the Adopt (n=12n = 12) and Abandon (n=9n = 9) groups were too small to detect anything but the largest effects.

Taken together, Phase 2 and Phase 3 indicate that AI use—measured as a binary variable or as an adoption trajectory—does not by itself explain essay quality. Prior writing ability remained the dominant predictor across both phases. This motivates examining how students engaged with AI, which we turn to in Q3.

Q3: Critical Engagement With AI

Q3 examined how students engaged with AI suggestions under the scaffolded Phase 3 workflow. The results indicate high levels of critical engagement in both self-reported practices and markers’ assessments of submissions, even though AI use did not independently predict essay scores.

Self-reported modification and verification. Students reported high levels of responsible AI engagement across both essays. Rejection or modification of AI suggestions was reported by 81% of AI users in Essay 1 and 84% in Essay 2; reference verification by 93% (E1) and 95% (E2). Neither change was significant, indicating a ceiling effect: students may already have understood that AI output requires verification, or may have reported responsible practices because the assessment foregrounded disclosure.

Result 3a (Self-reports showed high modification and verification): Rejection/modification was reported by 81% (E1) and 84% (E2) of AI users, and reference verification by 93% and 95%.

To understand why students rejected or modified AI output, we inductively coded the open-text explanations of why students rejected or modified AI’s suggestions (E1: N=275N = 275; E2: N=263N = 263). The most common reasons were protective of the students’ own work rather than mechanical: preserving their own ideas, argument, or meaning was the largest theme (E1: 39%, E2: 34%), followed by maintaining personal voice and writing style (E1: 20%, E2: 21%). A second cluster reflected the verification the scaffold sought to encourage: factual inaccuracy or hallucination (E1: 13%, E2: 10%) and citation problems such as fabricated references (E1: 9%, E2: 11%). Students described these judgments in their own words:

  • I modified some AI suggestions because some AI ideas were different from what I wanted to argue.

  • [It] made my writing sound too ‘robotic.’

  • My opinion is much better than AI’s.

  • It told me one of my sources was wrong. It wasn’t. It just hallucinated.

These responses suggest the high rejection rates reflect deliberate, criterion-aware judgment rather than indiscriminate refusal.

Result 3b (Rejection was deliberate, not indiscriminate): Open-text reasons for rejecting or modifying AI output centered on preserving the student’s own ideas and voice (the largest theme) and on catching inaccurate or fabricated references.

Marker-coded engagement. In Phase 3, markers applied three new codes to both essays: AI tone, critical engagement, and genuine insight. The clearest change was in AI tone (Table 4): Positive ratings increased from 47% in Essay 1 to 66% in Essay 2, while Negative ratings fell from 9.1% to 0.8% (both p<.0001p < .0001). Critical engagement (81% to 77%) and genuine insight (85% to 81%) were already high and did not change significantly. Overall, markers found that students engaged with AI suggestions rather than accepting them uncritically.

Result 3c (Widespread critical engagement; AI tone shifted positive): Markers coded 77–81% of submissions as showing critical engagement and 81–85% genuine insight. Positive AI tone rose from 47% to 66%, and Negative fell from 9% to under 1%.

Taken together, the Q2 and Q3 findings clarify a distinction between engagement quality and outcome quality. AI use did not independently predict essay scores in either phase, yet under the Phase 3 scaffold students modified, verified, and critically evaluated AI output.

CodeValueEssay 1 #Essay 1 %Essay 2 #Essay 2 %pp-value
AI TonePositive18147%23566%p<0.0001p < 0.0001
Neutral13936%9426%
Negative359.1%30.8%p<0.0001p < 0.0001
Did not use AI246.3%236.4%
Critical EngagementYes30981%27577%p=0.20p = 0.20
No4111%5616%
Did not use AI297.6%246.7%
Genuine InsightYes32785%28981%p=0.09p = 0.09
No5615%6719%

Table 4. Phase 3 marker-coded AI engagement indicators across both essays. N=383N=383 for Essay 1 and N=358N=358 for Essay 2.

Lessons Learned

This entire study was inspired by a deceptively simple question: Why wouldn’t students simply use AI when freely permitted?

Three years of data suggest a more interesting answer than they will or they won’t. AI use is shaped by policy and assessment design as much as by tool availability; many students engage with AI selectively rather than uncritically; and AI use alone does not explain writing quality. The clearest differences under scaffolding were behavioural, changing how students used AI rather than their scores. The lessons below summarise what we found and what we recommend:

Lesson 1 (AI Adoption): AI adoption depends on assessment framing, not only tool availability.

Under stable voluntary-use policy in 2024 and 2025, AI adoption held steady at roughly two-thirds of students. In 2026, adoption exceeded 90%, but this coincided with an expected-use and structured-workflow policy. Adoption does not rise automatically over time; it responds sharply to how instructors frame AI use in the assessment.

Lesson 2 (AI Insufficiency): AI use alone does not improve essay quality.

In both Phase 2 (unscaffolded) and Phase 3 (scaffolded), ANCOVA showed no independent effect of AI use on essay quality once prior writing ability was controlled. The strongest predictor of Essay 2 score was always Essay 1 score. The binary question Did the student use AI? is too coarse to explain writing outcomes. With several markers, some grading inconsistency is likely, which makes small effects harder to detect; this finding therefore indicates no large AI effect rather than no effect.

Lesson 3 (Scaffolding Effects): Scaffolding shapes practice, not scores alone.

Under the Phase 3 workflow, students showed high levels of modification, verification, and critical engagement, and AI tone became more positive between essays (47% to 66% Positive). AI use still gave no detectable score advantage, possibly because scores were near the ceiling (about 14.8/16) and AI was used mainly for grammar and style, while most marks reward reasoning, evidence, and reflection. This is the study’s central pedagogical insight: engagement quality and outcome quality are distinct goals, and scaffolding may advance the former without improving the latter.

Lesson 4 (AI Usage Pattern): Students mainly use AI as a writing scaffold.

Across all three years, students most often used AI for form-oriented support: grammar, style, structure, clarity, and organisation. Many students did not outsource ethical reasoning to AI; they treated it as a writing assistant that helped preserve their ideas more clearly.

Lesson 5 (Workflow Adoption): Keep the core scaffold; adapt the rest.

The core elements are the draft-before-AI step with its checklist, mandatory disclosure, and a marked reflection comparing draft and final essay. Topics, word limits, weightings, tools, and referencing style can be adapted, and the engagement codes are needed only for evaluation. For CS educators, this offers a low-cost way to assess ethics within a technical course. Full materials are available at [15].

Limitations and Scope of Claims

This experience report has several limitations. First, as a repeated cross-sectional study, it cannot track how individual students’ practices evolved. Second, it was conducted in one first-year course at one institution, so the findings may not generalise to other contexts, course levels, or assessment types. Third, the analysis relies largely on self-reported AI use, which is prone to recall and social desirability bias. Under-reporting is less likely in our study as AI use was permitted every year and expected in 2026, disclosure carried no penalty, and Phase 3 markers’ coding of non-use (6–8%) matched self-reports (7%). Self-reported verification is more prone to over-reporting, so we treat it as supporting evidence only.

The Phase 3 redesign also limits causal interpretation. Both 2026 essays used the scaffolded workflow, so there was no unscaffolded control, and cross-year differences mix design effects with cohort effects. Phases 2 and 3 used different score scales, so their grade-linked findings are comparable only conceptually. The engagement codes lack reliability testing, and high Phase 3 scores suggest ceiling effects. We therefore treat the findings as formative evidence of how scaffolding shaped students’ AI practices, not as causal evidence that it improves essay performance.

Conclusion

This experience report examined the use of GenAI for ethical essay writing over three years in a large first-year computer systems course. Across 1,894 survey responses, adoption held at about two-thirds under voluntary use (2024–2025) and exceeded 90% under scaffolded, expected use (2026). Students mainly used AI for writing support, and tool use diversified beyond ChatGPT.

In both grade-linked phases, neither AI use nor adoption trajectory independently explained essay quality, supporting evidence that use/non-use comparisons are insufficient [18]. The clearest Phase 3 findings were instead behavioural, with students reporting high levels of modification and verification, and markers saw widespread critical engagement and genuine insight.

The value of GenAI in writing-intensive computing assignments therefore appears to lie less in permitting its use than in designing structures, such as draft-before-AI workflows, checklists, reflection, and disclosure, that help students evaluate, adapt, and take responsibility for AI suggestions.

References

  1. [1]Andrea Ale Tubella, Marçal Mora-Cantallops, and Juan Carlos Nieves. 2024. How to teach responsible AI in Higher Education: challenges and opportunities. Ethics and Information Technology 26, 1 (2024), 3.DOI
  2. [2]Sasha Barab and Kurt Squire. 2016. Design-based research: Putting a stake in the ground. In Design-based research. Psychology Press, 1–14.
  3. [3]Alex Barrett and Austin Pack. 2023. Not quite eye to AI: student and teacher perspectives on the use of generative artificial intelligence in the writing process. International Journal of Educational Technology in Higher Education 20, 1 (2023), 159.DOI
  4. [4]Noelle Brown, Benjamin Xie, Ella Sarder, Casey Fiesler, and Eliane S Wiese. 2024. Teaching ethics in computing: a systematic literature review of ACM computer science education publications. ACM Transactions on Computing Education 24, 1 (2024), 1–36.DOI
  5. [5]Cecilia Ka Yuk Chan and Wenjie Hu. 2023. Students’ voices on generative AI: Perceptions, benefits, and challenges in higher education. International journal of educational technology in higher education 20, 1 (2023), 43.arxiv.org/abs/2305.00290
  6. [6]Rudrajit Choudhuri, Ambareesh Ramakrishnan, Amreeta Chatterjee, Bianca Trinkereich, Igor Steinmacher, Marco Gerosa, and Anita Sarma. 2025. Insights from the frontline: GenAI utilization among software engineering students. In 2025 IEEE/ACM 37th International Conference on Software Engineering Education and Training (CSEE&T). IEEE, 1–12.arxiv.org/abs/2412.15624
  7. [7]Jeet Dos. 2025. A Systematic Review of Research on ChatGPT in Higher Education. European Educational Researcher 8, 2 (2025), 59–76.
  8. [8]Montathar Faaron, Kari Rönkkö, Marcelo Milrad, and Eric Tsui. 2025. International perspectives on artificial intelligence in higher education: An explorative study of students’ intention to use ChatGPT across the Nordic countries and the USA. Education and Information Technologies 30, 13 (2025), 17835–17880.
  9. [9]Hieke Keuning, Isaac Alpizar-Chacon, Joanna Lykourentzou, Lauren Beehler, Christian Köpke, Imke de Jong, and Sergey Sosnovsky. 2024. Students’ perceptions and use of generative AI tools for programming across different computing courses. In Proceedings of the 24th Koli Calling International Conference on Computing Education Research. 1–12.arxiv.org/abs/2410.06865
  10. [10]Amruth N. Kumar, Rajendra K. Raj, Sherif G. Aly, Monica D. Anderson, Brett A. Becker, Richard L. Blumenthal, Eric Eaton, Susan L. Epstein, Michael Goldweber, Pankaj Jalote, Douglas Lea, Michael Oudshoorn, Marcelo Pias, Susan Reiser, Christian Servin, Rahul Simha, Titus Winters, and Qiao Xiang. 2024. Computer Science Curricula 2023. Association for Computing Machinery, New York, NY, USA.DOI
  11. [11]Barbara Moskal, Keith Miller, and LA Smith King. 2002. Grading essays in computer ethics: rubrics considered helpful. In Proceedings of the 33rd SIGCSE technical symposium on Computer science education. 101–105.
  12. [12]Aastha Pant, Simone Spiegler, Rashina Hoda, Jeremy Yoon, Naebeeb Yusuf, Tian Er, and Sheny Hui. 2024. Teaching Software Ethics to Future Software Engineers. In Proceedings of the 46th International Conference on Software Engineering: Software Engineering Education and Training. 391–401.arxiv.org/abs/2312.11884
  13. [13]James Prather, Paul Denny, Juho Leinonen, Brett A. Becker, Ibrahim Albluwi, Michelle Craig, Hieke Keuning, Natalie Kiesler, Tobias Kohn, Andrew Luxton-Reilly, Stephen MacNeil, Andrew Petersen, Raymond Pettit, Brent N. Reeves, and Jaromir Savelka. 2023. The Robots Are Here: Navigating the Generative AI Revolution in Computing Education. In Proceedings of the 2023 Working Group Reports on Innovation and Technology in Computer Science Education (Turku, Finland) (ITiCSE-WGR ’23). Association for Computing Machinery, New York, NY, USA, 108–159. doi:10.1145/3623762.3633499DOI
  14. [14]James Prather, Juho Leinonen, Natalie Kiesler, Jamie Gorso­n Benario, Sam Lau, Stephen MacNeil, Narges Norouzi, Simone Opel, Vee Pettit, Leo Porter, Brent N. Reeves, Jaromir Savelka, David H. Smith, Sven Strickroth, and Daniel Zingaro. 2025. Beyond the Hype: A Comprehensive Review of Current Trends in Generative AI Research, Teaching Practices, and Tools. In 2024 Working Group Reports on Innovation and Technology in Computer Science Education (Milan, Italy) (ITiCSE 2024). Association for Computing Machinery, New York, NY, USA, 300–338. doi:10.1145/3689187.3709614DOI
  15. [15]Asma Shakil, Paul Denny, Jun Seo, Gauravkumar Hirani, Sean Longyu Ma, and Claus Brabrand. 2026. A Scaffolded Workflow for GenAI Use in Ethical Essay Writing. doi:10.5281/zenodo.23882369
  16. [16]Jessie J Smith, Blakely H Payne, Shamika Klassen, Dylan Thomas Doyle, and Casey Fiesler. 2023. Incorporating ethics in computing courses: Barriers, support, and perspectives from educators. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1. 367–373.DOI
  17. [17]Elizabeth Towell and B Thompson. 2004. A further exploration of teaching ethics in the software engineering curriculum. In 17th Conference on Software Engineering Education and Training, 2004. Proceedings. IEEE, 39–44.DOI
  18. [18]Kaixuan Yang, Mladen Raković, Zhiping Liang, Lixiang Yan, Zijie Zeng, Yizhou Fan, Dragan Gašević, and Guangliang Chen. 2025. Modifying AI, enhancing essays: How active engagement with generative AI boosts writing quality. In Proceedings of the 15th international learning analytics and knowledge conference. 568–578.arxiv.org/abs/2412.07200

Paper details

vlnc:W_xqjne6hz
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 18

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents